Feature Portal
@sara as you asked for it
1. Votes disappear
Happened many times. Not only we noticed it. So you end up having 40 votes on a feature request and then, suddenly you go down to 29. Not only happened once, after we noticed and tracked it. Seems to be mostly Wednesday where the magic happened.
The disappearance of votes is like saying “our own customer survey showed something different”. Very intransparent.
2. Votes not counting:
2.1 from the same IP not counting
Of course related to 1., but shows a big problem. Let’s assume we have 500 users (only internal), 50 of them are trying to be involved and asking and demanding changes/feature requests to ensure a smoother and more stable and automated daily business, while working with CMK. So, 50 people who heavily using CMK and would vote, for there features
To quote the company behind the voting portal:
We throttle voting by IP address combined with the “User-Agent” text sent by your browser with each HTTP request. You may be surprised to hear that we allow multiple votes per IP address. The limit is very low. We allow this so that a person can ask one or two relatives, friends, or colleagues to vote up their submission. This lets the user feel they are gaming the system while in fact, they are having little impact on the final vote.
I like the last sentence especially. 
So imagine we are a company with over 500 active CMK users, with dozens of departments and only three of them can vote or would count?
As a CME user, we have in addition to that dozens of customers, who also request a lot of the features we create requests for. Same goes here as well, only 3 votes,doesn’t matter if we have 50 customers?
They are buying monitoring from us and expecting its working and asking us to make sure, more features are implemented – should we really bring there attention to the portal and even to the forum?
2.2 “Bulk voting”
Imagine you are in a meeting an presenting the feature portal to your coworker, as a way, we as a company have a “chance” to get a feature we desperately need.
Now everyone opens up the website and starts voting - but hey, there is a limit as written by the company behind it.
We throttle voting by suggestion. If a suggestion is receiving far more votes in a short timespan than is normal, it is probably manipulation. So we will silently ignore votes. Our throttling has several levels, including per minute, per hour, and per day limits.
Yeah, time wasted again for all involved. 
And you only know it if you are starting questioning what you noticed over weeks - reference 1.
3. Balance/value between votes from Enterprise Customers vs. RAW Edition
Related to 1. & 2.
The point was already made directly after the conference – I think it was @mike1098 in the community call
Paying customers, who are making sure CMK keeps alive and will move on in the future, have the same impact on votes as the ones “getting it for free”.
From a paying customer perspective, this does not feel fair.
Some more examples
- Enterprise vs. RAW
- DAX (Deutscher Aktienindex) / SIM Swiss Market Index Company’s vs. Handelsschule Kirchhellen (no offence! )
- Heavy User (1mio+ Services) vs small User (5k Services)
- Infrastructure driven vs. Application driven
To name another example as there been a couple of UX Topics lately:
A customer with 10 users has the same weight as a customer with 500 users where, a GUI improvement has a much greater impact.
4) Fake / manipulated votes
Also mentioned and tested in the forum before, that it’s possible to manipulate the votes – Thanks @simonm
So how can we really rely on the outcome of this votes? Furthermore, well noted by tribe29 half a year ago
“this is a problem we will have to think of and should solve.”
But where are we standing? When will it be solved? Is it on the Roadmap?
5. expectations vs. reality
Mentioned on the conference and also clearly by tribe29s PM.
“Ensure we work on features most important for the entire customer base”
“top voted = “candidate” for next release”
5.1. deciding what’s going to be implemented next
Of course it’s your right to decide what to implement next, but also creates an illusion for many of us.
Let’s take the first page of the Feature Portal. 50 Features, all about 30+ votes but only one implemented and only 6 scheduled
12% = planned
2% = implemented
86% = waiting feature requests by customers
For a customer, it does not really feel like “Ensure we work on features most important for the entire customer base” comes to life.
So to quote tribe29s PM
From my experience, there will be a long tail of request, which did get a single vote from the creator. Those will be closed. After 2 years, requests with 2-3 votes will be closed, or votes from a single company. It is all about keeping the portal healthy and useful
This sound more like we going to have a LONG tail of highly voted and needed features where the community, who is willing to help (and many even to pay) you with this features to make CMK even greater, will wait years for implementation, if even.
(Sidenote here, imagine you want to move on with with your monitoring, so what do you do if you cant accomplish it with the tool you prefer, start working with the ones who can help you solving problems in your daily business. Specially hard when your know it would be all possible with CMK - It still has the potential to get close to a “Eierlegende Wollmilchsau” of Monitoring tools)
5.2 fully understand the need of a feature request.
“Enable us to interact with customers to fully understand the underlaying need”
“PM clarifies requests”
14/50 (28%) Interactions with customers regarding the feature requests. Only considered the the first page of the Feature Requests Page, getting worse as less votes on features (will match with 6.) Correct me if wrong, but most likely because you are not even considering it? Patterns are there.
6. Human interaction vs. features to be recognized and upvoted.
Also recognized by tribe29 about half a year ago.
Also the interface of the portal is also quite influential, e.g. in the default view the highest voted suggestions are listed at the top. Guess what gets the most attention ;-).
As there are plenty of books about human interaction and how simple we actually are, same goes for our browsing behavior. There are plenty of papers written about it but to shorten it up, nearly nobody looks on search result “page 2” on google.Same goes with the feature portal as the amount of features is growing and growing. 1 implemented vs. 40+ new.
To quote your PM:
After 2 years, requests with 2-3 votes will be closed, or votes from a single company.
It is all about keeping the portal healthy and useful
If there are no people who are actively, checking the newest features and upvoting them, so they will end up on the “trending” page, they will just vanish with hundreds of others, sometimes really creative feature requests on some of the last pages and nearly no chance for someone to get it upvoted.
With 2-3 pages, not a problem, but with 12 pages already?
7. Time consuming
Who has the time for that? As mentioned already in this thread, it feels like we are doing a lot of QA and losing a lot of time with bugs, debugging and finding workarounds already. So to make sure, we as CME Customer can move forward and open up new markets, we need to check all feature requests and vote for everything which will make CMK better towards our needs and at the end to our customers? With an outcome of how much percent of our feature requests actually getting implemented?
8. Transparency about the process/status of feature requests
Here we jump back to the post from @PhilippL regarding the Public bug tracker and transparency.
@PhilippL also mentioned missing transparency regarding the Feature Portal. Feature requests since May with many votes are still “under consideration”
Nobody knows what that means (and feels nothing is happening). Some assumptions:
- It does not fit in your Roadmap
- tribe29 does not have time for that
- tribe29 thinks it’s not relevant
- It’s too time consuming to implement, you are focusing on a small thing (quick win)
…
Here we cross point 5.2 - first impression, nothing happening.
In fairness, IF you look into details and actually open up the feature request, sometimes you find at least some feedback from the PM. But still the status stays the same and often, even with engaging PM, we don’t know much more about it. If it’s going to be made in the first place and IF, in which release it will be implemented.
9. When do we know it’s the time you decide, what’s going into the next release?
This goes straight back towards point 8. - transparency
There is no “last call for votes for 2.2 release” or as far as I know, it seems now the call for the 2.3 release already started. We as customers have expectations and hopes that some of them of our feature requests will be implemented in the 2.2 release or even in the 2.3 release, but we don’t have any feeling and understanding and insides on how, when, and IF - nobody knows.
10. Reach - How many customers are using it really?
Isn’t it more or less similar to the forum, where just a minority of your customer base is actually using it? I’m not sure, was there a newsletter advertising it? If so, super! because you will reach definitely more.
Let’s assume we had the Conference where you announced it with 500 Customers (direct/stream) and then the Community Call with about 300 views on YT (so let’s assume 50 who didn’t heard about it before)) So due to the Forum we will put some extra, let’s make it 800 you reached. We have then maybe for many feature requests the same problem like with the “deprecation of service tags”, “nobody” uses? Just some thoughts to think about.
For example: https://features.checkmk.com/suggestions/299422/show-netapp-volume-efficiency#comment472115
I heard there is already a Partner who developed this, long time before the Feature Request was made, but nobody knows – not available to the public.
So imagine how much interaction you are obviously missing, in the feature portal from your customers
Possible in this case:
- Not one of them who are using it was willing to share the information
- They don’t even know/care about the feature portal and the request it self
From customer perspective the Feature Portal is for sure! an upgrade to the Forum solution “we had” before. And even if most likely not even 20% of your customer base are using it shows how much potential CMK has and how many fantastic feature requests users are coming up with to help and make CMK getting ahead of their competitors and make it a more sophisticated Monitoring tool as it is already.
But I’m not sure what you expected from it? I already said at the conference that I expected 300 votes till the end of the year (boy was I wrong!) and that it does feel more like a way of marketing to engage customers again and feel them being part of the product, like it was for many of us as they saw CMK growing. So far there was not much convincing us different. For every implemented feature, 40+ new feature requests open up. Huge potential, clearly, but like with everything, people losing trust and interest if nothing happened.
I truly hope that “Ensure we work on features most important for the entire customer base” will become reality again in 2023!