The Rampant Inaccuracy Of Fountain Pen Reviews

[Mister5]

I never relay on a singular review. I'm reading as much as possible before I'm making an opinion. Then I'm comparing photos that I can measure my own pens against the one reviewed. As well viewing videos when they exist. After that I'm still not convinced that I will like the pen, I will go away for a while to see if I'll find interesting couple of months later.

 

But that's me.

Ditto. I'll go through multiple reviews before biting the bullet. I'll look for "wisdom of the crowd" rather than one glowing review.

 

Most of my problems have come from vintage pens, and that's not something a review would really be able to address... there's always a dice roll with those unless you go to a pen show or get it from a reputable dealer. I've had one major problem with a new pen where the nib tines were visibly misaligned, and I could fix it myself.

[JakobS]

Though the rating system is an attempt to present a review in an easy to understand form, it does seem to become meaningless when many pens are rated near perfect, or one person's 7 is another person's 9, or 5. But, I do try to take the most holistic veiw based on the reviewer's experience, expectation's and biases. These are often easy to discern, or understand through their other posts on FPN, or elsewhere.

 

I will say, that the one pen review I have found time to post, on an Esterbrook SM Deluxe in 2010, is a pen I carry and use with me everyday. It's been seven years, and I still enjoy everything about it that I presented in the review.

[pajaro]

It's like Officer Effectiveness Ratings used to be in the Air Force. If you didn't have have perfect top marks and walk on water, you would sink into the mire and be forgotten, or they would send you to do something that might get you killed.

 

The pen reviews of new pens would be often done by someone who might or might not evaluate the pen coolly and rationally. The true nature of the pen might take a while to become apparent. I have personally been initially enthusiastic about many pens, and now most of them gather dust. Longer term usage and comparison with pens that meet or exceed my expectations might not give those pens a pass.

Edited March 9, 2017 by pajaro

[pepsiplease69]

I don't know that anyone is purposefully out there churning out 'dishonest' reviews.

 

Would a reviewer want to do that for a handful of reviews because maybe they are in cahoots with some retailer/manufacturer only to lose credibility for themselves.

 

Once you lose people's trust and credibility in your own brand, it's very difficult to earn it back.

 

Most of the reviewers are putting in time and effort of their own volition to make reviews with no other motivation than to get information out there maybe gain more readership along the way. Inaccuracy and dishonesty will be the quickest way to lose your audience.

 

And as to the comment regarding whether it should be called a review or a preview, I will say: the easiest thing to change is yourself, the hardest is to change others. You'll live much happier if you are able to adapt to things (still trying to fix in some way, what is obviously not right).

[cellmatrix]

Totally agree with the OP, far too often reviews are very unrealistically positive. No, its not some kind of purposeful conspiracy, pepsiplease. Instead, its simply human nature. Its about people lusting after a particular pen, getting it, and then showing off their grail catch in a review before they have had a chance to use it for a while, and only then realize its not as great as they had dreamed it would be.

 

As a consequence, the reviews just don't mean much to me. Plus FPN discourages negative reviews (remember no ink bashing), but does nothing to people who post unrealistically positive reviews. I think it all adds up to more profits for the vendors around here (and hey we do have some nice vendor folks, so I don't begrudge them from earning a buck). However this comes at the cost to the poor folks who come to FPN trying to find objective reliable information about a particular pen or ink.

 

Anyway, its just the way it is and its the way it will continue moving forward. We can rant about it, but its not going to change things. I've talked to Wim about it and you can too if you like. Wim is a pretty cool guy but one thing he can't do is change human nature. Instead, I think the best way to deal with the overly positive reviews is to wait a while and carefully consider the validity of any negative comments that might arise about a particular pen later in the thread.

[TSherbs]

Totally agree with the OP, far too often reviews are very unrealistically positive. No, its not some kind of purposeful conspiracy, pepsiplease. Instead, its simply human nature. Its about people lusting after a particular pen, getting it, and then showing off their grail catch in a review before they have had a chance to use it for a while, and only then realize its not as great as they had dreamed it would be.

 

As a consequence, the reviews just don't mean much to me. Plus FPN discourages negative reviews (remember no ink bashing), but does nothing to people who post unrealistically positive reviews. I think it all adds up to more profits for the vendors around here (and hey we do have some nice vendor folks, so I don't begrudge them from earning a buck). However this comes at the cost to the poor folks who come to FPN trying to find objective reliable information about a particular pen or ink.

 

Anyway, its just the way it is and its the way it will continue moving forward. We can rant about it, but its not going to change things. I've talked to Wim about it and you can too if you like. Wim is a pretty cool guy but one thing he can't do is change human nature. Instead, I think the best way to deal with the overly positive reviews is to wait a while and carefully consider the validity of any negative comments that might arise about a particular pen later in the thread.

 

I find much valuable objective information in both the positive and the negative reviews. It's easy to see this once you learn how to quickly sift through opinion and more objective fact.

 

Ultimately, the only criteria that count are the ones we subjectively interpret ourselves with our own experiences.

 

But one can find important information in both glowing and damning reports. And both kinds do exist here on FPN.

[praxim]

I may have said something along these lines earlier. Any review worth reading will include descriptions, photos and writing samples. If we are very fortunate the reviewer will even say what they like and dislike in general terms, their prejudices. What does a score matter, when you can review from the material yourself?

 

Whether for inks or pens, I read as many as I can find, and work out who makes sense and how it fits with my preferences. Many other people here have said the same. Shallow reviews are poor; in-depth biassed reviews can be interpreted and compared. No-one imagines this is high science.

[ac12]

MY opinions.

• An initial review is potentially very different from a long term review.

• A small problem that was ignored in an initial review may become a major irritant in a long term review.
  • The idea of the 'honeymoon' period is an example of this.
• People seem to adapt, sometimes quickly, sometimes slowly, or sometimes not at all.

• A problem that occurs in a long term review may not occur in the initial just inked review.
  • The fact that ink X CLOGS badly in the pen after 3 days would not be known in a "just inked" initial review. Though is this a pen problem or an ink problem, or both?

• One major problem that I cannot determine from most reviews is how much PRESSURE they are putting on the pen.
  • This is especially important when talking about how "flexible" a nib is.
  • If a nib flexes, but a LOT of pressure is required, then for ME, it is NOT a flexible nib.
  • But how much is a LOT? That is where the problem lies, as it is different for every person.
  • Even normal non-flex writing this has an effect.
    • My wife writes with more pressure that I do. So her ink line is wider and wetter than mine. And she may feel a pen more scratchy than me, as more pressure = more friction = scratchy feel.
• As has been said, "FEEL" is difficult or impossible to evaluate, because it is so personal.
  • I like slim pens, some people HATE them. So how do you evaluate this?
    • You need the absolute of the measured diameter.
  • I like light pens, some people do not like like light pens, and like HEAVY pens, which I HATE. Again how to evaluate this.
    • You need the absolute of the weight of the pen.
  • How about how smooth the pen writes. VERY difficult as it is dependent on many variables. And what is smooth to one may not be smooth to another.
• If a pen has to be adjusted to make it write well, then in my book, that NEEDS to be disclosed.
  • The problem is that most people do not have the skills to adjust the nib. So for them they would use it as it is (poor condition) or have to send it back (which costs $ and time).
  • If this is viewed as a QUALITY CONTROL problem, this warrants a ding against the max score. Or maybe a QC category/measurement has to be included.

Edited March 9, 2017 by ac12

[cellmatrix]

 

I find much valuable objective information in both the positive and the negative reviews. It's easy to see this once you learn how to quickly sift through opinion and more objective fact.

 

Ultimately, the only criteria that count are the ones we subjectively interpret ourselves with our own experiences.

 

But one can find important information in both glowing and damning reports. And both kinds do exist here on FPN.

 

I agree with you, there is still something to be had from a biased review if you learn how to read between the lines. You and I can do this as we have read a lot of reviews here and know what to expect. Its a learned process.

 

In contrast, someone new to the site expecting purely objective information who takes every glowing pen review at face value is surely going to be disappointed at some point.

 

I expect that many of us have pens sitting around in disuse, that we bought early in our fountain pen hobby at a time when we did not know any better and took overly positive reviews at face value.

 

Live and learn....

[ac12]

Do people recall how the Lamy 2000 seems to have generally GOOD reviews and comments, so as to be a commonly recommended pen.

 

Well, after seeing all the good recommendations, I got one, off the internet.

But, after 3 months of "trying to like it," I deinked the pen and put it into my collection, never to see ink again.

For ME, the 2000 was too FAT and too HEAVY. To me, it felt like an oversized Parker 45, and I do not like oversized pens.

This is an example of how "feel" is different between people.

 

BTW, the "ears" to hold on the cap did not bother me at all.

Edited March 9, 2017 by ac12

[TSherbs]

Do people recall how the Lamy 2000 seems to have generally GOOD reviews and comments, so as to be a commonly recommended pen.

 

Well, after seeing all the good recommendations, I got one, off the internet.

But, after 3 months of "trying to like it," I deinked the pen and put it into my collection, never to see ink again.

For ME, the 2000 was too FAT and too HEAVY. To me, it felt like an oversized Parker 45, and I do not like oversized pens.

This is an example of how "feel" is different between people.

 

BTW, the "ears" to hold on the cap did not bother me at all.

Yes, but there were a lot of hohum reviews and even "this pen sucks and is over-rated." truly, in 5 years here I haven't seen what I would call a favorability bias. The reader also has predilections which must be self-monitored. Sometimes we want to like a pen too much even before we own it to justify the purchase impulse.

[pajaro]

I think that if you have experience using some different pens, you will be able to separate the wheat from the chaff in reviews. Most of the posters here have some very pointed opinions, and you are unlikely to be endangered or misled by the reviews. Reviewers are telling you what they think, which is what has been posted in this thread.

[Arkanabar]

We can't all be Consumer Reports. (Incidentally, their review of fountain pens found a LOT of misaligned nibs, back in the day.)

The point is so that people who can't handle these pens get some sort of an idea of what the pen is like. When I read a review (or watch a video) I don't care one little bit if the person loves or hates the pen. As far as I'm concerned the good or bad scores are meaningless. I'm not buying a pen for the guy/gal who made the review. I'm buying it for me. So if the review mentions a texture or weight, I ignore their judgement and think about if I would like that texture or weight. I'm looking at the pen for what I think I would like or dislike about it. Not everybody is going to love/hate the same things for the same reasons for any amount of time. I used to love this or that, but now I don't so much, and vice-versa. So I don't think it's that the reviews are dishonest, it's that our tastes and opinions change over time, which I think is a good thing. I still don't like the Pilot Metropolitan though.

 

Now, if a retailer paid somebody to put up reviews of pens saying how awesome they all are, and didn't disclose that fact, then THAT would be dishonest.

This. Whereas watch_art likes to look for the objective information, I do my best, on those occasions where I review a pen, to provide as much objective information as possible. Just as all probability is conditional, all opinions are biased. I also do my best to be informative and objective about the ink and paper used. But I don't try to "objectively" grade stuff; I tend to provide descriptive rather than numerical impressions of a pen's characteristics.

I also don't do video reviews. I'm happy to share opinions and information, but I'm not one to perform for the Internet. I usually don't watch them, either.

Then there is the hurry to be first.

I do tend to hold off of instant reviews. I prefer to wait a while and make pretty good use of a pen before reviewing -- more than one fill, at least. One common cognitive bias is our love of new and shiny stuff. And if we're going to tout the durability of fountain pens, we ought to be able to provide some idea as to whether anything important wears out -- for example, the cap on my Baoer 388 flighter is becoming dangerously loose.

Just read as many reviews as possible. Eventually a clearer and more realistic picture of the pen emerges.

 

+1. 'nuff said.

[Uncial]

I tend not to put scores into any reviews I do anymore simply because I think they are so subjective and bit random. I don't really know what I am supposed to be comparing it to. What might be a ten out of ten nib for me might be a boring, overly smooth butter mush for someone else. So, I leave it out. There is a large degree of subjectivity to any review also. There are many things that go into an individuals thoughts on any particular pen and with an expensive pen there might be a desire not to see the faults initially which explains why they sometimes don't appear in reviews that are clearly done in the honeymoon period. There are also times when I've bought pens that everyone else seems to adore and I just haven't bothered doing a review because I'm conscious that my personal preferences make me the odd one out. The flip side of that is that when it's an expensive pen I feel there's a certain duty to do it so others don't get stung if their preferences happen to match mine.

 

In regards to no negative reviews here on FPN, I remember questioning this some time ago really to keep myself right more than anything else. As far as I remember the consensus was that ranting and venting with overblown claims of awfulness was probably not in anyone's best interests. Making a claim about a pen as fact (in negative terms) for all types and models on the basis of one experience with one pen when every other review is more positive, perhaps indicates that something is awry. Negativity about a bad experience and/or a really badly made pen is ok......I think, and that all seems fair enough to me.

[tonybelding]

This topic is spurned by a discussion on /r/fountainpens. Something I've seen across the board whether they are reviews on this website or from the most popular YouTube reviewers is an overall inaccuracy and at times even blatant dishonesty in regards to reviewing pens. Primarily this issue manifests as pens being reviewed too soon after they are bought. This is why 99% of the reviews in our review Forum are at or close to 10 out of 10. We often get rose-tinted glasses from the newness of the pen and thus our reviews are not completely accurate. I say we because I have done it a few times myself.

 

Another big issue I have seen across the board is the rather absurd case of a pen being raved about and being given perfect scores meanwhile there is the ' minor detail' that the reviewer had to take micromesh to the nib and/or realign the tines to make it write well. I'm sorry but a pen's single purpose is to write. If a pen is unable to write well out of the box then it is a bad pen. Period. There is this ridiculous notion that it should be standard practice to have to adjust a pen out of the box. But just because it is commonly needed doesn't make it any more excusable. It just means we have enabled companies to consistently give us bad products.

 

One of the nice things about FPN is that we can come back later and update our reviews with additional comments, which often results in a much clearer picture. This has happened to me a few times, when I had to come back and report problems, like the parts breakage on the TWSBIs or the persistent flow problems of the Karas Fountain-K -- or even the fact that I finally got a usable nib (third nib's a charm!) for my Wahl-Eversharp Skyline Technic.

I have to disagree a little about nibs needing to be adjusted, though. It's just one of the sad facts of fountain pen life. Nobody has ever (yet) invented an automated way to adjust nibs, and the skilled labor required to do them all by hand is prohibitive. You can get all huffy and call it ridiculous, inexcusable, a bad pen and so forth until you're blue in the face, but none of your words or your outrage are going to change the way things are. It's baked into the nature of the technology.

[marcelo]

Did you not say

 

So are you asserting that really is the case? If not based on your personal experience of receiving pens where you felt some need to align tines or use micromesh what is that based on?

 

You definitely seem to have made a claim that reviews are dishonest, not that they are inaccurate but rather dishonest. Is that not your claim?

 

You go on in a later post to claim

 

Again, if it was not disclosed in the review how could you know it happened? If it was disclosed in the review then where is the dishonesty? As I pointed out and asked (and you even quoted) I'm not sure how you determined any review was not honest? Are you objecting to the fact that the reviewer did not follow those procedures you believe are required? Are you objecting to the fact that peoples tastes and personal preferences change as they gain experience and over time? Are upset that people change their minds about a given subject over time?

 

Where is the inaccuracy, much less dishonesty?

Agreed! 👍

[ethernautrix]

You can't protect the world.

 

Life is full of risks and surprises. I made plenty of pen purchase mistakes; I blame no one - not the reviewers, not the tantalizing photographs, not the admiring crowds - but myself. If I don't know from the beginning how to assess a pen review, then I will learn from my mistakes.

 

But everyone who lives in a consumerist culture ought to have learned something about marketing and manufactured desire and how to cut through the subjective or how to spot where one's own subjective aligns with the subjective in the review.

 

A few years ago when I was one of the most ethusiastic Nakaya rah-rah girls (including guys, too), I became aware that people were tempted to spend a large amount of money on a single pen based on my raving about my new favorite brand. So I started saying, Nakaya isn't for everyone. Don't be afraid not to like it. At least you'll be able to sell it easily at almost no loss.

 

I guess that's the risk we have to take, as buyers, with any pens, especially when we don't have an easy way to check the pens out with our hands before we buy them.

 

Having said that, yeah, I have experienced the honeymoon period with some pens, but I don't review pens, so.... It's funny, cos my honeymoon period with Nakaya didn't begin until almost two years after I bought my first one. They're still my favorite -- I just love the urushi. Now if I could transplant a Pilot PO nib into one of my Piccolos....

[CrispyBacon_87]

There are a lot of people in this thread being triggered by the word "dishonest".

 

It was a poor word choice by the OP, but I think it's meant as "not true" the or "not accurate" as opposed to "deceitful" or "lying". Nobody is going around writing fake reviews (at least not on this website). The OP clearly meant that people are writing reviews that are not true representatations of the pens, whether it's due to lack of experience, the honeymoon phase, etc...

 

I do agree. The people going around giving 10/10 for every pen are not posting anything useful. Whether it's a $20 pen or a $2000 pen, there's probably always something that could be changed or improved.

[TSherbs]

The OP clearly meant that people are writing reviews that are not true representatations of the pens, whether it's due to lack of experience, the honeymoon phase, etc...

 

I do agree. The people going around giving 10/10 for every pen are not posting anything useful. Whether it's a $20 pen or a $2000 pen, there's probably always something that could be changed or improved.

I think that this misrepresents both the frequency and meaning of "10/10". First, no one gives every pen a 10; and second, any rating is just a subjective measure of a personal experience. Even when someone gives empirical measurements, the rating is not much more than an appreciation meter. These ratings are just subjective shorthand, like Matt Armstrong's "feedback scale." Don't we all know this?

[praxim]

If people want fully accurate reviews then we could limit them to a complete set of weights and measures, photographs taken at 90° to top, bottom and each of four side angles, and a statement of whether the ink goes in, stays in when not in use, and comes out when used.

 

While that information is very useful and mostly necessary, this is a review site for people who love their pens. I would be happy for no scores ever to be given, especially if each review included at least some comparative comments with other pens the person uses. That is pretty much the format I plan for my next effort at a review: two to four pens of the same brand or class included, data, qualitative ratings rather than scores, and comparative comments on use with my conscious biasses stated.