Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Ask PG: How has voting habit/volume changed since being hidden?
144 points by scorchin on April 23, 2011 | hide | past | favorite | 74 comments
It's been over a week since you started experimenting with HN [1]. It would great to hear some feedback on what significant changes (if any) this has caused to the voting habits of the user base.

[1] http://news.ycombinator.com/item?id=2434333

Edit: Seems that PG answered a similar question recently http://news.ycombinator.com/item?id=2465002



I personally do not like the fact that the voting process is being hidden. I hope PG will take it away soon. These are my reasons:

1. From my experience, I stopped voting because I am not being educated on the fact that we need to vote (well I can't see the rating, so why vote). I have a feeling that other users act similar. Please confirm.

2. The voting process shows the values of HN. Comments that are ok in techcrunch would be downvoted here within a second. And I feel that voting system needs to be open to educate new hackers & founders about what's acceptable and what's not. I know there are rules but we learn by practice.

3. When voting process is open, good comments get even more votes, and irrelevant or bad comments get even more downvoted. I personally don't want to read irrelevant posts. I have limited time but now I feel like going through comments that really don't have enough information.

PG, I hope you take my personal opinion into consideration. Thanks


In regards to #2, I still think hiding low-value comments in some fashion would be the way to go. For example, how about hiding comments that fail to receive X upvotes within Y days? (And, of course, an obligatory link to display all comments, or perhaps all comments in a sub-thread)

Something simple like (for example) a comment needs 1 upvote per day to remain unhidden, until some number of days pass or some number of points are achieved (4? 5?), and then the comment remains visible in perpetuity.

I like the new no-points system, but I do agree #2 is a concern and could used some tweaking.


Yes. I agree that we need to hide some comments that really don't add any value.

On the side note, I got more votes right now than ever before. Go figure how people think about voting o_O


<badcomment>Agree</badcomment>


So every down vote is a +1 for bringing back the vote count display, right? Or was the subtlety that thick?


It's a bad comment even though it's only being bad ironically.


My personal observation has been that, while blind-folding might have been good for curation of the comments, by avoiding the bandwagon effect, but it has serious ill-effects on the readability, especially for the new subject matter for the reader.

In the past, there were 2 signals of the comment-quality in the thread.

1. Vote count.

2. Positioning in the thread.

Now, that Vote count is not visible, There is only one signal left.

1. Positioning in the thread.

It makes it very difficult to segregate good comments from the bad ones, and decide which comments add to the topic, and hence worth reading/discussing, especially on a large thread, on a new subject matter, reader does not understand very-well.

Few suggestions to improve the comment readability of the thread.

1. Would it make sense to hide the comment count of a thread till a particular threshold (say votes < 7, comment time < 10 mins), or of the entire thread comments (thread time < 30 mins, number of comments < 30) and the likes, and make them visible to add to the readability/visibility of the thread?

2. Would it help if the up-votes are capped?


I feel that there is more than simply impact on the readability but also on the semantics.

When I feel clueless about a topic I look to comments with up votes to get a general feel for the direction of the discussion and hints how to educate myself on the topic.

On your first suggestion: Maybe it is hard to work with absolute thresholds as the amount of votes is probably proportional to how long something stays on the front page. Maybe there is some measure to determine when a thread has begun to "settle" down and that would be the time to make votes visible.


There was always a huge first-mover advantage to getting a high scoring comment... so I think your #1 above was actually a misleading signal.


>There was always a huge first-mover advantage to getting a high scoring comment

It's right for good comments that are read by more people to receive more votes. Also that the first person to make a particular comment should be higher voted as repetition of the same insight isn't improving the thread.


It's right for good comments that are read by more people to receive more votes.

I don't agree with this. Just b/c someone was first to post a mediocre comment doesn't mean it's good or worth reading.

Also, early posts are often knee-jerk and among the least thoughtful of the thread.


> Now, that Vote count is not visible, There is only one signal left. > 1. Positioning in the thread.

I'm not sure if positioning in the thread is much of a signal. Comments with a low vote replying to a comment with a high vote will get displayed above comments with a medium vote, and will more likely be read. Before, the reader could choose to ignore such comments, and keep on scrolling down to check for other comments with a high enough vote.


What about adding greenness to comments? Comments which are in order are black no matter what, but if a comment is above another comment with more upvotes because it's newer then it's slightly green to indicate the disparity in votes.


To me, the site has gotten a lot less informative and interesting since the votes have been hidden. The voting numbers really help me understand what people are thinking: how strong the consensus is, how much support there is for dissenting opinions. Without that data, at least for me the value of the discussion here is significantly reduced.


I would have t agree. There was ask HN recently where someone was asking for advice. The advice I would give was already posted so I up voted, but now the guy doesn't know if the advice had 100 up votes and really well regarded or if he just got really few responses. I could have made a yeah what that guy said comment to make my agreement more visible but that is rather frowned upon.


I'd guess that "yeah what that guy said" comments won't be as frowned upon with the new system.


But how does that affect the "Decline of HN?" I would think that threads filling up with comments where the summary of the content is a "+1" or a "-1" detracts from the rest of the conversation thread. Especially with something that would have normally gotten 100 "me too" upvotes.


Who cares how many points a given comment have gotten? Who cares what other people think? I see voting as a way to bury the clearly trolling or off-topic comments, not as a way to take the temperature of some community.

Maybe you get twenty-three down votes and forty-six ups; the point count isn't going to tell you that. I know that I have comments where the score hovers around one but have maybe ten votes that cancel each other out. What to make of that?

A voting system should have modest goals. (While the Slashdot community hasn't been my cup of tea for a while, I feel like they've muddled their way to a system that works pretty well.)


For example, as I write this, I have 23 upvotes for the parent comment. How does that compare to some of the others? More than the other top-level comments, presumably, but what about the nested comments? Since voting patterns have changed, past experience isn't any guide.


I was going to upvote that comment until you told me how many votes you have -- now I feel like it's probably been upvoted enough.


I agree and here is my solution: two axes of moderation. The big, conspicuous, open-to-everyone up and down vote arrows feed into an agree/disagree poll whose total (agree votes minus disagree votes) is displayed by the comment.

Meanwhile, the useful/useless votes are smaller or perhaps harder to access, like the current "flag" system for comments where you have to click through to the comment's own page first. Perhaps voting either agree or disagree automatically registers your vote for useful, and to vote useless you have to click through like with "flag".


Agreed. Not for all content though. But for what's foreign to me. Ideas, concepts, even products. If I don't "get it" right away, comments help guide me to a better understanding. Lower scores indicate noise. (Not really, but I won't know the difference.) Sorting comments without points isn't enough though; you don't know when exactly the noise begins and where to stop reading.

It's even useful for topics I'm already familiar with. If a comment I don't necessarily agree with has a high score, I'll wonder why. It's a great opportunity to walk down new mental paths or explore different angles about the topic without feeling vulnerable. If any insights emerge while doing this, I'll then follow up with my own comment.

In either case, it's impractical to analyze or reflect on every comment, so points are a pretty good heuristic. Even if it is a broken system, it does help a lot of the time.


It seems one of the main complaints in the current format is the lack of a quick visual guideline to contrast comments amongst each other. Maybe PG can create a numerical ranking system for comments similar to the way stories are ranked on the homepage. It would quickly show readers how informative each comment is in comparison to another without having to reveal comment scores.


Sorry guys, but YC interviews started today so I'm busy all day for the next week and a half. Maybe after that I'll write a little code to do some analysis.


Many people seem to be saying they've been voting less. Could you please include a script that counts total weekly votes prior to the change and post-change to see if the total number of weekly votes has shifted downwards?


Crude tests suggest there is about 20% less voting.


This seems like a good thing for comments. The goal for comments is civility, not democracy.

Actually, the goal is for the comments to be interesting, but they will be because there are interesting people on HN. If interesting people are leaving, then that's a problem.

I feel that the tone of comments in general has improved significantly. Maybe that's confirmation bias, but I feel much more "at home" on HN now.


It's still a democracy if high-scoring comments float to the top though. Karma just adds to it by functioning as a reward system.


My personal totally unscientific observation is that I seem to upvote less often. On the other hand, I seem to be getting slightly more upvotes than before the change.


I upvote more because I never think, "well, I'd just be piling on at this point". And it's more based on agreement or appreciation instead of righting karmic injustice.


That seems to be in line with pg's response in the linked thread with the same question.


My knowing that the owners of the site step in and try to fix problems causes me to invest more time and attention on voting because it reduces the probability that that my time and attention will go to waste due to incompetence or inattention by the owners.


Hell, until this thread, I didn't even know you could still vote! I assumed something was broken since the scores weren't visible and have been waiting for them to reappear.


Since scores have been hidden, voting on a comment feels less like a contribution and more like talking to myself. I vote less. It seems like one dimension of the community -- an admittedly imprecise form of collective intelligence -- is missing. It's certainly not a dealbreaker, but the comment threads feel ever so slightly "flatter".


Agreed - I'm voting less. Voting seems to have less 'point'. Previously it would feel like voting was (in a minor way) helping the community, by helping to point out to others comments that deserve reading. Now only the comment writers see the score, it feels voting up is only to 'reward' the writer with a bit of karma. To me it devalues what karma is about and makes it feel very selfish. Its lost its value to the community. Whilst karma influences the top comment, this doesn't seem enough to make it valuable enough to vote regularly.


Having done some redesigns of HN to see my own reaction towards the concealment of scores and other elements I have to come out in full support of concealing comment scores. After playing with a layout that concealed the scores for comments and stories I'm also a big fan of ditching score display for stories.

My experience was what I think common sense dictates. Since I couldn't see a score I couldn't let that color my opinion of a given comment and had to read it. This point alone has significantly increased my engagement in the threads I care to read and made me more carefully consider the top level comments on a story. Since pg turned off comment scores I've wondered if the primary reason people have reacted negatively is due to losing a positive status indicator. This is to say the score might have nothing to do with one's own engagement with HN, but with one's concern for other user's perception of their comments.

One other change I experimented with was moving the score of stories and comments to the far right top of the story/comment so that it wasn't the first thing in my line of sight. This helped with my bias as well.

At the end of the day this all anecdotal, and your mileage may vary.


I can't explain why but it's pushed me away from commenting.


Is that a bad thing? I personally think the greatest threat to HN is not bad comments, but too many mediocre comments. I think the best case scenario is if only the people who had a lot of knowledge or things to say about a given thread commented. It's equally easy for HN to be dragged down in a comment deluge as it is for comments to simply become worse.


Thank you. One of the implicit assumptions of user-generated content sites is that every user should be given a voice. Unfortunately, this creates feedback loops around certain topics (around here, it tends to be Linux/C/vim) because there are too many benefits attached to discussing those topics; namely, karma and [illusory?] in-group status.

One idea is to give each user only a finite number of comments they can make in a time period. As their karma increases, so does their comment allotment.


One idea I really liked from the previous discussion on this was to hide the Names of all comments for a few hours.

This way comments really can be voted on their merits rather than the fan club of the commenter.


This could become very confusing on new comments with several layers of response/conversation going on. That said, I like the spirit of the idea.


Perhaps the names on a particular thread in the discussion could be revealed once you vote or comment on that thread?


I'm going to vote against this: I personally never look at names anyway. Eyes jump between comments themselves, and only if a person is marked as a friend do I lift my eyes up in a "oh yeah, it's you again" tilt.


If you never look at the names, what does it bother you if the names are or are not shown?


Because it's nice to be able to quickly find out what the name is once I do.


I'd be interested in seeing how we'd react with a logarithmic voting system.

Base 10 for example: the most-significant digit would be visible and the rest be zero's: '5' -> '5', '42' -> '40', '256' -> '200'.

That way, you could still see the impact of a comment while hiding the voting trends and other numeric details that keep you from unbiased (err, less biased) voting.


It might be more mathematical to use e=2.718 rather than 10, and perhaps even displaying the log value instead of the rounded number, e.g. 3 -> 1, 9 -> 2, 29 -> 3, with suitable fiddling for negative vote totals.


Why is it more mathematical to use e? (Hint: depending on what you're trying to do, it's not.)


Might be interesting, if much more computationally intensive to show the z-score of a comment vs the rest in the thread (that have votes).


Just to toss out an idea that addresses some of the complaints I've seen: you could always reveal the total score after votes are placed (and you could replace the score with a question mark until then, clicking on which would indicate "I don't want to vote" and would reveal the score). If hiding names is under consideration, you can solve the thread confusion problem by replacing names by letters in a consistent way. Again, real names could be revealed by clicking, removing the ability to vote. The main problems I see in this are when revisiting threads later (now you cant vote on new comments) and the possibly dominant strategy of beginning every post with an allusion to "my bingo card software".


For me the amount of votes did matter - it gave you a quick idea as to where some of the most important comments were within the thread. Also HN seems to having far lesser activity - this someone with actual data might want to check and get back


No votes showing up, is bad for "Ask/Show/Tell HN" type of submissions, where the submitter is asking to see what "wisdom-of-the-community" says (even if he has the entire knowledge for subject matter).

For example, asking for feedback or feature suggestions on the recently developed product to the community, will not be much helpful, if comment votes are not shown up.


I have a few ideas, but first I will say that I like the hidden karma. 1) Show karma after a vote. I feel this would assuage the claims that voting has no meaning or impact. 2) Order threads by replies/length. Not foolproof, but in many instances longer comments and threads are more meaningful and relevant. 3) Karma visible when not logged in, hidden when logged in.

Again, just ideas, I'm sure they can be improved or refuted (and I'd much enjoy the discussion). If you feel a comment is inaccurate or misleading, reply to it and explain. The reply will tell more than a number will.


I'm not sure I care about comment scores, but I am concerned by how much other people seem to. Relying on scores to choose which comments to read is like getting all your news from the most watched cable news channel. I might prefer that the comments be sorted chronologically.


In being a participant in many of these online communities, these kind of things sometimes seem like a losing battle to me. There will always be a slice of the population that will try to game the 'karma' system on sites like HN. Slashdot went thru this years ago. Kuro5hin went thru it. Digg. Reddit.... All sorts of solutions are proposed, like hiding a user's 'karma' rating, having the user population meta moderate ratings on comments...

I do find it interesting that the comment score has been hidden. That's kind of novel, and it might be the kind of solution needed in this case.


This is an interesting question. Thanks for the link here to a recent comment by pg on how karma scores are looking on various individual comments.

In sizing up what this means, let's all review together the stated reason for the experiment, expressed in a thread-opening post by pg titled "Ask HN: How to stave off decline of HN?"

http://news.ycombinator.com/item?id=2403696

He wrote, "The problem has several components: comments that are (a) mean and/or (b) dumb that (c) get massively upvoted."

How do the highest-voted comments visible in the bestcomments list

http://news.ycombinator.com/bestcomments

look to all of you recently? Are there fewer mean comments than before? Are there fewer dumb comments than before? Are the comments that are "massively upvoted" since the experiment began mostly comments that are reasonably kind and well-informed, helpful comments on the whole? In most of the treads you visit, do helpful, thoughtful comments seem to rise to a position of prominence, while mean or dumb comments gray out?

A link and comment in the thread referred to by the edit on this submission largely sums up the back-and-forth about visible comment scores as a signal on comments in active threads:

http://news.ycombinator.com/item?id=2465357

>> Please bring back the comment scores. It helps a lot in parsing the comments and assigning a proportional weight to each when reading them.

> I had to think about this a bit, and I disagree so far. I'm finding that I'm not pre-judging comments as much. It's nice to be able to read someone's comment without knowing first that 70 or 80 or 3 other people thought it was worthwhile.

My impression too is that even with comment scores not visible, it is still convenient to browse threads to find thoughtful, informative comments, but now there is less anchoring bias

http://www.sciencedaily.com/articles/a/anchoring.htm

of most votes on a comment converging to one score level that shows up early in a thread's development, and more engagement by readers of HN in actively reading comments and upvoting (or downvoting) based on each comment's characteristics in light of the context of the thread.

But the main criterion of the experiment is to "stave off decline of HN," and that is what will decide if the experiment was successful. For that every reader can help by actively upvoting informative, helpful comments, and also by downvoting comments that are either mean or dumb--and especially comments that are both. As I recall, the experiment has also involved some changes in the effects of flagging, so flagging inappropriate comments is also helpful.


  but now there is less anchoring bias
I think this is a large one. I've been participating more in discussions lately because the content seems more thoughtful.

  How do I know what's important without vote counts?
Read and decide what is good and what is bad. For posters, being concise helps people reading. The higher voted comments should move to the top like in the past(?) Forcing both sides to be more thoughtful makes the quality content and participation better.


Agreed, I think quality has improved, and either by placebo or by reality, I dig it :)


There was a similar thread couple(?) days back, PG replied. Search.


Yup, here's the thread: http://news.ycombinator.com/item?id=2465002


As someone who's been here from a while but not from the beginning, I find myself relying more on name recognition. For example, on this thread I saw tokenadult's, grandalf's, praptak's and criticsquid's comments first because I had a recognition of quality based on their history of commenting.


Two ways the changes have impacted my browsing:

1) Removing "by" at the start of each comment stops me from trivially finding every other comment by the same poster in a given thread. Searching for just the name adds noise to the results. Usually not a lot, but I find it annoying.

2) Removing comment scores from our thread feeds makes it much more difficult to skim through all the things a given user has said recently looking for gems. Instead of a nice pre-filter (skim for posts exceeding some minimum score) you have to read every comment, no matter how inane.


I agree with several statements here that the HN lost some information level since the removal of the karma. One more voice here would like to see some statements from PG and how the change affected the community. If there is a donation button to get the karma back I would contribute.


PG might consider doing something similar to what I am doing on Hubski (http://hubski.com). -Scores have ranged values, only the user sees their absolute score.

Maybe I can give something in return for the great source code. :) Thanks PG.


My given comment score average has plummetted yet bizarrely my actual average comment score is higher than the average before the alteration (due to one or two over-regarded comments).


Unfortunately the changes have broken one of my most used Android apps -- HNDroid. I hope either the changes here revert, or the app developer updates it, cuz I miss it! :(


Either get rid of the whole Karma business or allow everyone to both upvote and downvote. Anything in between is just a hypocrisy.


I would be really interested to know if anyone is looking at the comment posting velocity of users before the change and after.


It's harder to figure out what's worth reading.


I have a suggestion: make it an option and give users the choice to view vote counts.


HN is going in the shitter these days. Negative karma for posting an on-topic suggestion. Well, it was fun while it lasted.


I've started (just now) using a stupid '+1' comment because I thought the reader should be looked at.


Hey, I appreciate you +1'd my comment but it's better not to do that. The ideal on HN is for every comment to be worth reading. If you want to draw attention to a worthy comment and are frustrated the votes are hidden, just contribute something to the discussion. The subthread will grow and more readers will notice the comment and your reply to it.


...notices that they have been hidden...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: