The Importance of Peer-Review, or How to Spot a Huckster.  

Hucksters are individuals who claims to have all of the answers; that “main stream” industries are corrupt profiteers out to harm you or your pet’s health for financial gain; and that only be listening to them (the Huckster) and/or by buying their products will you and your pet live long happy and healthy lives. So basically everyone else is lying/conspiring except for them. Sounds a bit to me like Kaa from the Jungle Book as he tries to wrap Mowgli up for dinner, “trussssst in me…”.

Mowgli and Kaa the snake

How I feel about Hucksters…

Even if you believe that large corporation (be it food or pharmaceutical companies for pets or people) are solely driven by profit, how much long-term profit is there in harming your customer?  Small corporations become large corporation because they provide a service or product that people (the ones with the wallets) want to purchase. There are numerous business books written on this very topic. Corporations with any financial and business sense want those wallet holders buying products for as long as physically possible. Their financial incentive is to prolong health and longevity not shorten it.

In the world of pet food and pet care, there is no shortage of people claiming to have all of the answers and a “study” that was recently “published” online that was too good (and by “good” I mean really bad) not to weigh in on.

Huckster Red Flag: Presenting “findings” through media outlets instead of peer-reviewed journals.

It is important to understand that anyone can write a book or develop a website on any topic they wish. There is no requirement that they need to be an expert in a particular field (or even have any training in a particular field) to voice their opinions loudly and publicly. When I started my consulting website and this blog I didn’t have to prove my credentials before I started. I paid my website registration fee (website) or filled in a registration form (blog) and off I went. I told you my credentials and experience and many people will take me at my word and never look further, but I could easily have made it all up. Fortunately, my education and experience are easily verified as are those of my Veterinary Nutrition colleagues, yet self-taught and self-proclaimed nutrition “experts” still seem to hold more sway than actual experts when it comes to matters of pet food and supplements. Forget the scientific process, these individuals want you to believe them because they say so.

evidence-pyramid i just know

This recent posting from The Spudd (thespudd.com) sums up my take on pseudosciences.

Where I think opinion starts to cross the line into pseudoscience is when experts (real or imagined) wrap their opinion in charts and science-y sounding words in an attempt elicit maximum emotional response, but without providing any actual results or following any scientific method to speak of. They use smoke and mirrors, misdirection, and misrepresentation of “findings” to tell the story they want believed instead of informing and educating the readers. I’d like to think that individuals who perpetuate this type of pseudoscience started with a genuine concern for wellness and wellbeing, be it human or animals, but have got themselves so entrenched in their beliefs that they lost sight of reality.  So how can you as the concerned pet caregiver tell fact from fiction? One way is to read everything with a healthy dose of skepticism.

In the world of true scientific publishing, the peer-review process helps act as a critical eye before studies are printed. This process is not perfect and occasionally bad or biased studies are released, but overall the peer-review process is an important step in ensuring the integrity of published science. When studies are submitted to a peer-reviewed journal, the entire materials and methods used by the researchers are outlined in detail. This allows the reviewers to examine and critique the study looking for bias in methodology or omission of important details, and whether the researchers conclusions are supported by the finding. Some journals have started requesting the raw data so that the reviewer can look for irregularities themselves or even repeat statistical analyses. If these reviewers miss key details and a paper is published with gaps in methodology or with conclusions not supported by the data, the journal’s readers can write to Letters to the Editor outlining their critique of the article in question. The researchers then have a chance to respond or even reconsider their stance. Sometimes published research papers are revised and re-published and sometimes fatal flaws are reveal and the paper is removed from reference libraries (just like the retracted and discredited “study” linking vaccines to autism).

“Scientists will hold up their research for critical evaluation while Hucksters will hold up the fact that something is printed as evidence of it being true.”

If a company or individual prints their study on a website, or in conference proceedings (meaning that the researcher or a representative of the researcher talked about their finding at a meeting somewhere), or in marketing materials, the critical review stage has been by-passed completely. The messy details of methodology and actual data are typically omitted and we as the reader have no knowledge of how the study was conducted or whether there are critical flaws that would invalidate the results. The reader is expected to accept the researcher’s conclusions as truth without the opportunity to review the methods or data, or even evaluate whether there are conflicts of interest between the researchers and those “publishing” the study. Scientists will hold up their research for critical evaluation while Hucksters will hold up the fact that something is printed as evidence of it being true. In my opinion true scientists are willing to admit the fact they don’t have all of the answers, which is why they keep asking questions.

cat choices

Testing the hypothesis…

What do I look for in a published research study? I want to know where the tests were performed, how they were performed, how many samples were used, what they used as positive or negative control (controls are when help ensure that environmental contamination is not occurring which can bias the findings), and what statistics were used (statistics help differentiate random chance from true relationships). A well written Material and Methods section will allow another researcher to replicating the study. This is how the field of science grows, by testing hypotheses and challenging assumptions. The absence of Materials and Methods and any statistical evaluation in any study (whether submitted for peer-reviewed or “published” online) are bright red, waving flags for me.

Not that pet food manufacturers or owners should be blasé about potential problems in pet foods, but without any study details we have no way of knowing whether the researchers grabbed one dented can on the shelf, did one fungal test and then stopped testing when they found a (weak) positive result, or whether they tested multiple times or different batches of foods in each brand and repeatedly found the same result. Even if you were to trust their laboratory procedures on blind-faith, the more detailed section on bacterial contamination admits that the bacteria found were the result of improperly refrigerated food. Yet in the infographic, the reader is presented with a table outlining how almost all of the products tested positive for bacteria. Could that also be true for the fungal toxins? Everyone (hopefully) knows that if you leave any food exposed to room air at room temperature it will start to grow things, but improperly stored and handled foods by the consumer is very different than poor manufacturing practices.

“improperly stored and handled foods by the consumer is very different than poor manufacturing practices”

Let’s look more closely at the reported risk from fungal contamination. Mycotoxins (toxic compounds from fungi growing in and on foods) are a serious problem in foods and ingestion of even small amounts per pound of body weight can causing serious health effects and even death in people, dogs, and cats.  Again, if we accept the tests were done correctly and the results were not a factor of improper storage and handling, when you read the details about the fungal finding it is only when you add all of their ‘low’ risk numbers together that the ‘risk values’ get worse. I haven’t taken a statistics course in almost 20 years, but I’m pretty sure you can’t just add numbers together to provide significance.

As for the ‘nutrient’ concerns, let me start by saying that dogs and cats do not have sulfur requirements and there is no lower or upper limited in the 2006 NRC or any version of AAFCO for sulfur because it is not an essential dietary nutrient. There are requirements for the sulfur-containing amino acids (methionine, cystine, and taurine) and sulfur is found in connective tissue, which is a part of any meat, so it is not surprising that it was found in the diet. (I would be more surprised if it wasn’t.) Excess calcium intake on the other hand is a problem in growing large breed dogs, but since the foods that tested higher in calcium were either cat foods or adult dog maintenance diets, neither of which should be fed to a puppy, I’m also not worried about these nutrient values.

This “study” also compared nutrients in foods on a Dry Matter basis without providing Calorie (kcal) contents of the foods tested, but direct Dry Matter basis comparisons between foods are problematic and not used routinely by small animal (or human) nutritionists. Dogs and cats (and people) will eat to meet a particular energy requirement each day while the mineral requirements are independent of energy intake. The calcium content of an energy dense diet should be higher than the calcium content of a low energy diets to ensure that the animal received enough calcium per day despite eating less Dry Matter. To put in simple terms, if a dog needs to eat 3 cups a day of Brand X dry food to meet energy needs, but only 2 cups of Brand Y dog food to get the same Calories, then Brand Y must add 50% more calcium per cup to ensure that the dog is receiving the correct calcium intake per day despite having a lower Dry Matter intake. Providing Dry Matter values alone is meaningless.

red flag

Veterinarians and caregivers need to be advocates for their furry family-members and access to accurate, unbiased information is an important part of that. So here is what I consider my Pet Food and Supplement Red Flags:

# 1: Results promoted through media and not scientific journals.

#2: Miracle “cures” that are not supported by the larger medical or veterinary fields.

#3: Proprietary or secret formulas or methods.

#4: “Proof” that is provided by testimonials instead of facts.

cat on computer

Education is key, but so is checking the references

Happy Feeding!

Lisa P. Weeth, DVM, MRCVS, DACVN

Advertisements

28 thoughts on “The Importance of Peer-Review, or How to Spot a Huckster.  

  1. Pingback: The Truth About Pet Food Research | Seresto Collars

  2. Pingback: The Truth About Pet Food Research | Angels Eyes Blog

  3. When cat and dog food is tested/followed on animals for a longer period than 6 months, then opinions on pet food will mean something. Until then, speculation on appropriateness of a diet is simply that, speculation. Unfortunately, it’s speculation regardless of how “learned” the speculator or how much money is thrown at it too.

    Like

    • Hi Anne,

      Thanks for reading and for your comment. I completely agree that a 6 month food trial in an adult won’t tell you anything about the long-term affect of that diet. It will show any major problems, but borderline deficiencies or excess can take years to develop. I have personally seen adult dogs and cats in practice over the years that were fed diets that I knew to be deficient in essential nutrients (e.g., fed only chicken breast and white rice for years), yet they walked around my exam room without any problem, and other potential coat problems, they looked outwardly fine. I know what the eventual clinical signs will be of the given nutrient imbalance, and that some (like thiamin deficiency) will show up in a few weeks, while others (like calcium deficiency) can take years. Growing cats and dogs are another issue, and nutritional problems will present themselves within a few weeks.

      I am an advocate of annual blood work and urinalysis in all dogs and cats (young, middle age, or senior) eating a home-prepared diet (even ones balance by a Veterinary Nutrition Specialist like me), and even for adult dogs and cats eating commercially prepared complete and balanced diets. I usually start monitoring blood and urine in the commercial diet fed dogs and cats at about middle age and I look for any abnormalities, but also trends. If I see that a given dog’s urine concentration starts to go down over the years without a change in diet then I will keep a closer eye on that dog’s kidney function tests, even if that value is still considered “normal” by the laboratory. One of my practice philosophies is that I would rather try to prevent or delay a medical condition, than try to fix it after the fact.

      When I speak at veterinary conferences or work with veterinarians in their practices, I also recommend that they collect diet histories on all of the dogs and cats they see. It is only by knowing how their patients do on specific diets that they will develop a better understanding of the quality (or lack thereof) of the commercial diets out there.

      -Lisa Weeth

      Like

  4. Pingback: The Truth About Pet Food Research | Blog for Discount Dasuquin

  5. Pingback: The Truth About Pet Food Research | Pet Care Blog Articles

  6. I find it strange that there should be no minimal sulfur requirements for dog or cat diets. Obviously, some of their tissues contain sulfur, and that sulfur has to come from somewhere, so it’s rather obvious that they do need to take in sulfur with their food. Do we just not know how much of it they need, or what’s the problem here?

    Like

    • Hi abb@cdd.ef,

      You are absolutely right that sulfur is a component of other things that would normally be found in the diet. Dogs and cats have a minimal requirements for the sulfur-containing amino acid, methionine, as well as a total sulfur amino acid (methionine plus cysteine) requirement, but there is not a separate and distinct sulfur requirement. This is also the case with cobalt and Vitamin B12 (also called cobalamin). Cobalamin is an essential vitamin for dogs and cats (and people!) and cobalt is an essential part of cobalamin, but cobalt itself is not required by dogs, cats, or people because we lack the metabolic machinery to make cobalamin. Similarly, you can leave elemental sulfur out of the diet completely as as long as there is adequate methionine+cysteine, the dog or cat will do well. Does that help?

      -Lisa Weeth

      Like

  7. Pingback: The Truth About Pet Food Research | Glyco Flex 3 Blog Page

  8. Pingback: The Truth About Pet Food Research | Pet Owners Care

  9. Pingback: The Truth About Pet Food Research | Pet Care from the Web

  10. Pingback: The Truth About Pet Food Research | Pet Care Around the Web

  11. Pingback: The Truth About Pet Food Research | Pet Health Articles Around the Web

  12. Pingback: The Truth About Pet Food Research | Pet Care Articles

  13. By what company or companies was your veterinary nutrition sponsored by? Do you consult for or act in any advisory capacity for any manufacturing companies?

    Like

    • Hi Tracy,

      Thanks for reading and taking the time to ask a few questions. My veterinary school training was self-funded and my residency was supported by the University of California, Davis, exclusively. Some Veterinary Nutrition residency programs do receive grants from outside corporations that cover the cost of training new residents and resident research (even though mine wasn’t funded this way), but even in those situation the outside company does not exert any influence on the training program itself or the recommendations made by the residents and faculty. During my 4 years of veterinary school and then nearly 13 years of veterinary practice (7+ as a board-certified Nutrition Specialist) I have attended lectures sponsored by veterinary food and pharmaceutical companies and have been invited to tour a number of commercial pet food manufacturing sites across the country, but despite the free continued education I always puts the needs of the patient and their family first.

      After my residency I went into private referral practice and worked with caregivers and veterinarians developed dietary treatment plans for dogs and cats irrespective of brand or form of food (I actually make made more home-cooked recipes than prescribed commercial foods). My experience with patients over these years has led a few companies to seek me out for advisory roles, but my industry experience has been intentionally limited for two reasons: 1) My first love in veterinary medicine is working with patients and caregivers, so I’d rather be in a clinic than in office tower; and 2) as an employee or spokesperson for a corporation you are expected to promote the company at all times, and I don’t think any one pet food company makes diets that are perfect for all cats and dogs or that any one diet or supplement company can promise optimum health. With that said, I have worked with a few veterinary companies (both large and small) as an outside adviser over the years. Most of the time it has been for one time reviews or advise, but I have served on a Nutrition Advisory Council for a medium-sized pet food manufacturer on a more on-going basis for the last few years. This has given me insight into pet food production and safety standards that I think compliments my clinical work, but doesn’t influence my recommendations or medical advise to caregivers. You won’t see me promote or recommend any specific brands on this site, and I have specifically blocked ads on this blog. I want to give caregivers and veterinarian unbiased tools to make decisions about what is best for their individual dogs and cats.

      -Lisa Weeth

      Like

  14. Pingback: The Truth About Pet Food Research | Learn More about Pet Care

  15. Pingback: The Truth About Pet Food Research | Pet Care Articles and Media Around the Web

  16. Pingback: The Truth About Pet Food Research | Pet Health and Well-Being

  17. Pingback: The Truth About Pet Food Research | Cosequin Pet Blog

  18. Pingback: The Truth About Pet Food Research | Blog for Discount Denosyl

  19. I’m not a veterinarian or nutritionist, but I do have an advanced degree in science (PhD, chemistry) and I’ve certainly had to deal with experimental design. Only six dog and six cat foods, some wet and some dry, were tested. All of them were national brands that you can get in a grocery, pet, or discount store, something TruthaboutpetFood.com discourages. (They have a “Find Healthy Foods” link). In the Pet Food Test Results” article there are many references to budget, as in “our budget did not allow for…” However, this website regularly offers reviews of what they (apparently?) consider to be superior products. Instead of just testing products they obviously had issues with, they should have waited until they had more money via crowdfunding and then tested some of what they considered to be superior, healthy products. In addition to everything you’ve pointed out above, whether the results mean anything or not, the people who commissioned this study cannot now point to the food they advocate and say that it DOES NOT have the same “stuff”, whether this “stuff” is dangerous or not. To call the study “in-depth” is laughable.

    Like

  20. I found a biography of Susan Thixton and there’s nothing in her background that qualifies her to discuss pet foods at all. She claims she became “an advocate” because her vet told her that her dog’s “bone cancer” was caused by “preservatives.” She also claims the vet told her the dog had bone cancer after she found a lump on the dog’s hindquarters – bone cancer usually affects the legs. I found out a lot more about cancer than I wanted to know when our 12-year old male Chow mix developed lung cancer and two months after we lost him our 10-year old Golden developed mast cell tumors that got into her lymph system. Nowhere have I seen anything about cancer being caused by “preservatives in food.” She told one interviewer that she called the manufacturer of the food she had been feeding the dog and asked what the shelf-life of the food was and was told “TWENTY-FIVE YEARS!” (it must have been canned food.) She’s making money by writing books and articles and selling subscriptions to her newsletter and other things. She’s a huckster, there’s no doubt about it. The “dog food advisor” is another one. It’s the blog of an oral surgeon who claims he lost a dog he adopted from a shelter due to “poor nutrition.” Neither of them have any kind of qualifications to make them the “experts” they represent themselves to be. The Dog Food advisor is sponsored by an online pet business in Florida and Thixton is supported by donations.

    Like

    • Hi Sam,

      Thanks for taking the time to look more closely at both of these self-proclaimed “advocates” and sharing your findings here. There are a few other infomercial-type sites out there that I wish I could warn owners away from, but alas they have bigger lawyers and deeper pockets than I do so I have to tread lightly.

      With cancer, so much comes down to genetics and age. When I was in practice in New Jersey I had a older couple bring their 4 month old Golden retriever puppy in to see me. Their main concern was in trying to prevent cancer in this specific dog over the course of his lifetime. Their previous 3 dogs (all Goldens) had died from cancer (the youngest and most recent loss had been a 2-year-old that developed lymphoma the year before they came to see me). They were hoping that some combination of diet and supplements would prevent the same fate from occurring in this new puppy. I told them unfortunately the best way to avoid a cancer diagnosis is by getting a different breed. Goldens have one of the highest risks of cancer development of any dog breed so our focus would be on minimizing preventable health risks, like development joint disease and obesity rather than cancer specifically. But there are plenty of people who would have sold them expensive bags of food and hundreds of dollars worth of supplements.

      Lisa

      Like

    • Hi David,

      Sorry for the delayed response. I just relocated back to the US from living in the UK and everything has been a bit slower these last 2 months.

      I don’t have favorite brands necessarily, more like top 5 favorite business practices. I tend to stick with companies that 1) work closely with nutrition experts (either PhD in Animal Nutrition or veterinarians with Diplomate status in the American College of Veterinary Nutrition (DACVN), or both) for diet formulations; 2) either make their own foods (own their own manufacturing facilities) or have direct oversight with a third-party manufacturer; 3) perform quality control testing on raw materials and finished products; 4) know the full nutrient profile of their diets (basically do more than the minimum requirement required by law); and 5) have good follow-up with questions and concerns. The World Small Animal Veterinary Association (WSAVA) has a good reference that can help guide evaluation of diets and companies (http://www.wsava.org/sites/default/files/Nutrition%20on%20the%20Internet%20dogs.pdf).

      Lisa

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s