Reliability Comparison Study
HP Color Toner Cartridges
vs. Remanufactured Brands
Cartridge Reliability
Print Quality
Color Accuracy
September 2005
For distribution in the Americas and Asia Pacific
5401 Tech Circle 鈥? Moorpark, CA 93021 鈥? 805 531 9030
6148 N. Discovery Way, Ste. 175 鈥? Boise, ID 83713 鈥? 208 424 1905
漏 2005 QualityLogic Inc.
Reliability Comparison Study
Executive Summary
During 2005, QualityLogic conducted a study for HP designed to test the quality, reliability,
and color accuracy of Hewlett-Packard (HP) Color LaserJet toner cartridges compared to
twelve brands of remanufactured color toner cartridges for the HP Color LaserJet 4650
(Q3668A) printer. The brands tested were from North America (6 brands), Europe (5
brands), and Asia Pacific (1 brand).
In the test, more than 670,000 pages were printed spanning all brands of cartridges. The
pages were printed in a continuous mode in a controlled environment and were designed by
QualityLogic to replicate the types of pages customers would typically print.
Cartridge Reliability by Brand 鈥? when combining all problem categories, an average of
80.1 percent of all remanufactured color toner cartridges tested exhibited reliability
problems, compared to 1.9 percent of the HP color toner cartridges tested (see Table 1).
Print Quality by Brand 鈥? based on pages randomly sampled, HP printed more than four
times more 鈥淗igh Quality鈥? pages than the average of all remanufactured cartridges tested
(see Table 2). A 鈥淗igh Quality鈥? page is defined as print quality level 4 or 5 as defined in
Appendix 1.
Color Fidelity - all remanufactured color toner cartridges tested demonstrated noticeable
color differences when compared to HP color (see Table 3).
Color Consistency - HP demonstrated better color consistency than all other
remanufactured brands (see Table 4).
Remanufactured Brands Tested
North America
Corporate Express
o
Elite Image
o
GRC
o
Image Excellence
o
Office Depot
o
OfficeMax
o
Europe
o K&U Printware
o Kores
o Lyreco
o Pelikan
o Supplies Team
Asia Pacific
o Best Jet
Page 2 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Test Overview
Cartridge print quality and reliability for each brand were determined by inspecting a 10%
random sample of the pages printed to determine the print quality (PQ) of the page. Each
page inspected was rated on a scale of 1 to 5 (with 5 being best) to determine the PQ Level
of that page. Pages classified with a PQ Level of 4 or 5 were considered 鈥淗igh Quality鈥? and
pages classified with a PQ Level of 1, 2, or 3 were considered 鈥淟ow Quality鈥? (refer to
Appendix 1 for details).
Test Pages
File: C1 File: C2 File: C3 File: C4 File: C5
Application: Application: Adobe Application: Application: Application: Adobe
InDesign CS Acrobat Microsoft Excel XP Microsoft Word XP Illustrator 7.0
Description: Cooking Description: White Description: Description: Description:
School Flyer Paper Cover Page Spreadsheet Whitepaper page Corporate Identity
Since a combination of four cartridges working together as a set for each brand was
required to print the test pages, the PQ Level of each of the 4 cartridge colors (Cyan,
Magenta, Yellow and Black, or CMYK) was recorded for each page inspected. This allowed
for quality and reliability comparisons by cartridge, as well as by total pages printed. This
number of 鈥淗igh Quality鈥? pages per cartridge per brand was then used for several
comparisons:
Cartridge Reliability by Brand- each problem cartridge was classified as Low Quality, a
Premature Failure, or Dead on Arrival (DOA). DOAs, Premature Failures, and Low Quality
cartridges were then combined to determine an overall problem percentage for each brand.
Print Quality by Brand 鈥? in this comparison, an overall percentage of 鈥淗igh Quality鈥? and
鈥淟ow Quality鈥? pages was calculated for each brand.
Color Fidelity 鈥? measures a brand鈥檚 ability to accurately reproduce a color. Using an XRite
Spectrodensitometer, a standard measurement instrument for hard copy output, a numeric
L*a*b* value is determined for each color measured. The difference between the swatches
measured for tested remanufactured brands and HP is expressed as a Delta E. The higher
the number, the greater the difference between the color fidelity of the remanufactured
brand compared to HP.
Page 3 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Color Consistency 鈥? measures a brand鈥檚 ability to print colors consistently. Using the same
process used for color fidelity, each brand was compared to itself on each printer it was
tested on throughout the duration of the test, and an average L*a*b* value was determined
for each color. Each measurement was then compared to this average brand value to
determine the average Delta E for this brand. The lower the number, the more consistent
the colors were throughout the duration of the test.
Page 4 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Results
Cartridge Reliability by Brand
On average, 80.1% of remanufactured cartridges had some form of reliability problem,
compared to less than 1.9% of HP cartridges.
Number Reman %
of Total Percent Problem/
Cartridges Problem Problem HP %
Brand Tested DOA PF LQ Cartridges Cartridges Problem
HP 52 0 0 1 1 1.9%
Brand A 33 2 7 22 31 93.9% 49.4
Brand B 31 3 1 22 26 83.9% 44.2
Brand C 32 2 0 19 21 65.6% 34.5
Brand D 32 0 2 24 26 81.3% 42.8
Brand E 32 4 6 21 31 96.9% 51.0
Brand F 32 0 4 20 24 75.0% 39.5
Brand G 32 1 7 18 26 81.3% 42.8
Brand H 32 2 4 16 22 68.8% 36.2
Brand J 32 1 0 27 28 87.5% 46.1
Brand K 32 4 6 21 31 96.9% 51.0
Brand L 24 0 5 19 24 100.0% 52.6
Brand M 32 2 0 9 11 34.4% 18.1
HP 52 0 0 1 1 1.9%
All
Remanufactured 376 21 42 238 301 80.1% 42.2
Brands
Table 1:
Cartridge Reliability by Brand
DOA Cartridges
A cartridge was classified as a DOA if it printed 10 or fewer 鈥淗igh Quality鈥? sampled pages
and fewer than 2.5% of the Adjusted Average Yield for like color cartridges from this
manufacturer.
For example, if a cartridge failed to print at least 10 鈥淗igh Quality鈥? pages but still printed
more than 2.5% of its Adjusted Average Yield, this cartridge was considered to be a
Premature Failure (having a high number of 鈥淟ow Quality鈥? pages) rather than a DOA. The
following is an example of the types of problems observed:
Severe toner leakage
o
Broken drums
o
Page 5 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
o Excessive lines and fading on printed pages
o Toner covering the entire page
o Loss of toner on page and drop outs
o Incorrect labeling of cartridges (cyan toner in a magenta identified cartridge, etc.)
o Broken pull strip (plastic seal broke off inside cartridge)
Premature Failure (PF) Cartridges
If the total number of pages printed by a cartridge was less than 75% of the Adjusted
Average Yield for like color cartridges of a brand, it was classified as a Premature Failure.
For example, if a cartridge printed a total of 3000 pages, but the Adjusted Average Yield for
that brand of cartridge was 5000 pages, this cartridge would be classified as a Premature
Failure.
The following are some observations that were made when cartridges failed prematurely:
o Toner leaking from the cartridge
o Anomalies on printed pages that became more severe over time
o Cartridge was deemed to be causing paper jams
o Supply memory errors
o Grinding noise from cartridge in printer
Low Quality (LQ) Cartridges
A cartridge was classified as Low Quality if it failed to print at least 90% 鈥淗igh Quality鈥?
pages, but was neither DOA nor a Premature Failure. The following details the most
common print quality problems observed in these cartridges:
o Fade 鈥? a slight loss in toner intensity that shows as lighter print areas and possible
color shifts. Typically occurs towards cartridge End-of-Life.
o Smudge 鈥? an unexpected or irregular amount of toner visible on the printed page
(front or back of page). Usually small, but length is measurable. In severe cases, this
can cover the entire page.
o Drop Out 鈥? a lack of toner causing loss of color or definition and color shifting.
o Uneven Fill 鈥搃mage density is inconsistent and can appear to be splotchy or mottled.
o Speck 鈥? a small spot or mark of toner visible in an unexpected location. Can be
repeating or a single occurrence.
o Line 鈥? a visible dark or light area of toner that runs in either the vertical or horizontal
plane. Can be repeating or a single occurrence.
o Color Shift 鈥? a visible change in color from an established standard.
Page 6 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Print Quality by Brand
HP cartridges printed more than four times the number of 鈥淗igh Quality鈥? pages than the
average of all the remanufacturers, based on the ratio of 鈥淗igh Quality鈥? to 鈥淟ow Quality鈥?
pages inspected for each brand.
High Quality Pages Low Quality Pages
% High
Brand PQ5 PQ4 PQ3 PQ2 PQ1
Quality
HP 13.2% 83.0% 3.9% 0.0% 0.0% 96.1%
Brand A 0.8% 0.7% 86.6% 11.4% 0.6% 1.5%
Brand B 6.6% 9.7% 62.9% 17.1% 3.7% 16.3%
Brand C 0.4% 33.3% 60.6% 5.3% 0.3% 33.8%
Brand D 4.5% 26.3% 57.3% 11.6% 0.3% 30.8%
Brand E 0.0% 3.2% 83.9% 12.7% 0.2% 3.2%
Brand F 0.5% 18.1% 74.2% 7.0% 0.1% 18.6%
Brand G 8.4% 9.3% 67.4% 14.9% 0.0% 17.7%
Brand H 3.8% 26.9% 65.7% 3.6% 0.0% 30.7%
Brand J 4.2% 16.1% 74.0% 5.7% 0.0% 20.3%
Brand K 0.5% 4.5% 91.3% 3.1% 0.6% 5.0%
Brand L 0.1% 1.7% 77.7% 19.6% 1.0% 1.7%
Brand M 6.0% 77.6% 16.3% 0.0% 0.0% 83.7%
HP 13.2% 83.0% 3.9% 0.0% 0.0% 96.1%
All
Remanufactured 3.0% 19.0% 68.2% 9.3% 0.6% 21.9%
Brands
Table 2:
Print Quality by Brand (Percent of Sampled Pages)
Page 7 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Cartridge Reliability Problem Examples
The following photos document some of the issues observed with remanufactured brands鈥?
color toner cartridges:
Photo 2: Brand K magenta - severe leak
Photo 1: Brand D - yellow toner leak
discovered when sealed envelope was opened
(classified as a Premature Failure)
upon receipt (classified as Low Quality)
Photo 4: Brand G - magenta toner leakage
inside printer (classified as DOA)
Photo 3: Brand B - toner residue on transfer
belt which left smudges on the backs of all
pages (classified as Low Quality)
Page 8 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Page Quality Samples
The following are examples of pages that were ranked as Print Quality (PQ) 1 and 2 pages:
Low Quality Page (PQ Level 1)
Excerpt from page sample to the right Yellow and Cyan cartridge spill onto white space
Low Quality Page (PQ Level 2)
Excerpt from page sample to the right Yellow cartridge fails to print in lower right
Page 9 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
The following is a sample of a PQ 3 page 鈥? note the repeating cyan dots down the left and
right sides of the page (circled in black):
Page 10 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Color Fidelity - Brand Color Comparison to HP
Color fidelity is the measurement of a color toner cartridge set鈥檚 ability to reproduce colors
compared to a benchmark. For this test, HP was used as the benchmark and each
remanufactured cartridge brand was compared to HP.
As shown on the table below, the lower value of the average Delta E means that the color
printed is closer to the HP color. Therefore, the remanufactured color cartridge set with a
higher Delta E measurement will more likely have a noticeable color difference from the
expected color.
All remanufactured color toner cartridge brands tested demonstrated noticeable color
differences when compared to HP color.
Delta E
Brand to HP
Brand A 6.44
Brand B 5.98
Brand C 5.88
Brand D 6.31
Brand E 9.03
Brand F 7.66
Brand G 6.30
Brand H 6.39
Brand J 6.39
Brand K 5.71
Brand L 7.18
Brand M 5.37
All 6.66
Remanufactured
Brands
Table 3:
Brand Color Comparison to HP - Average Delta E Measurements
(based on sampled pages)
Page 11 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Color Consistency
Color consistency is the ability of a color toner cartridge set to maintain consistent color
printing throughout the life of the cartridges. Approximately every 500 pages, the CIE
L*a*b* values for several color swatches were measured. From these measurements, an
average CIE L*a*b* value for each color was calculated to use as a reference for each
brand. This value was then compared to each measurement taken for a specific brand, and
the difference (Delta E) was used to determine how much variation there was in the brand
when compared to itself across several cartridges.
HP had the lowest variation - an average Delta E of 2.3 - compared to an average of 4.2 for
all remanufactured brands:
Mixed Pure Average
Colors Colors
HP 2.52 2.14 2.33
Brand A 5.71 3.78 4.75
Brand B 4.35 3.12 3.74
Brand C 5.56 3.32 4.44
Brand D 4.29 3.23 3.76
Brand E 6.60 4.46 5.53
Brand F 5.65 3.83 4.74
Brand G 4.48 3.08 3.78
Brand H 4.60 3.19 3.90
Brand J 3.62 3.74 3.68
Brand K 5.29 3.49 4.39
Brand L 3.78 3.39 3.58
Brand M 2.63 2.20 2.42
HP 2.52 2.14 2.33
All 4.90 3.51 4.21
Remanufactured
Brands
Table 4:
Brand Color Consistency Average Delta E Measurements
(based on sampled pages)
Definition:
Mixed Colors 鈥? CIE L*a*b* measurements taken on a target that uses a mixture of
pure colors
Pure Colors 鈥? CIE L*a*b* measurements taken on a cyan-only target, magenta-only
target, etc.
Page 12 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Appendix 1: Table of Definitions
Test Project Terminology Definition
Adjusted Average Yield The average of the total number of pages for each color cartridge by
brand from data collected during this test. Pages printed by DOA and
Premature Failure cartridges are not included in this average.
End-of-Life (EOL) When the printer required replacement of a cartridge before it would
continue printing or when the print technician observed serious
degradation of print quality.
High Quality Page Print Quality Level 4 or 5 as defined below.
Low Quality Page Print Quality Level 1, 2, or 3 as defined below.
Dead on Arrival (DOA) Cartridge A cartridge that printed 10 or less 鈥淗igh Quality鈥? sampled pages and
fewer than 2.5% of the Adjusted Average Yield for like color cartridges
from this manufacturer.
Premature Failure (PF) Cartridge If the total number of pages printed by a cartridge was less than 75% of
the Adjusted Average Yield for like color cartridges of a brand, it was
classified as a Premature Failure. DOA cartridges are classified
separately and are not included in this category.
Low Quality (LQ) Cartridge A cartridge that failed to print at least 90% 鈥淗igh Quality鈥? pages. DOA
and Premature Failures are classified separately and are not included in
this category.
Problem cartridge Cartridge exhibiting one of the problems listed above: DOA, PF or LQ.
Print Quality (PQ) Level 5 Page No apparent artifacts and a user would put this page in his or her
resume. Combined with Level 4 (below), these pages are defined as
鈥溾?淗igh Quality鈥濃?? pages.
PQ Level 4 Page May have a minor flaw such as one speck or uneven graphic rendering,
but the average user would still use it in a typical business document.
Combined with Level 5 (above), these pages are defined as 鈥溾?淗igh
Quality鈥濃?? pages.
PQ Level 3 Page Sufficiently flawed that it would not be circulated to others as a business
document and would only be acceptable as a draft page. This would
include noticeable levels of color shift. These pages are defined as 鈥淟ow
Quality鈥? pages.
PQ Level 2 Page Have lost some legibility and/or have a definite shift in color and would
need to be re-printed. These pages are defined as 鈥淟ow Quality鈥? pages.
PQ Level 1 Page Have lost content and/or have a dramatic shift in color and would need
to be re-printed. These pages are defined as 鈥淟ow Quality鈥? pages.
Delta E (鈭咵) Delta E is used to describe mathematically the distance between two
colors in a color space. To calculate the Delta E of any two colors, you
need to know their L*a*b* values. The Delta E is the square root of the
sum of the squares for the L*, a* and b* values. The larger the Delta E,
the larger the difference in the color.
Spectrodensitometer An instrument that captures spectrophotometric data and provides
consistent CIE L*a*b* color measurements.
Page 13 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
Appendix 2: Test Methodology
The test methodology for this reliability comparison study was developed by QualityLogic for
HP. The following is a summary of the methodology used for this study:
A total of 376 remanufactured cartridges and 52 HP cartridges were tested. On average, a
total of 32 remanufactured color toner cartridges from each brand (11 black, 4 cyan, 11
yellow and 6 magenta) were tested, and 52 HP color toner cartridges (18 C9720A black, 8
C9721A cyan, 16 C9722A yellow and 10 C9723A magenta) were tested. A total of 26 Color
LaserJet 4650 (Q3668A) printers were used in the testing (two for each brand). Only 24
Brand L cartridges were tested because QualityLogic was not able to procure needed
cartridges from the manufacturer. These sample sizes were selected using statistical
methods to achieve statistically significant results.
QualityLogic procured all printers, paper, and HP color toner cartridges through standard
retail channels. Remanufactured color toner cartridges were obtained through multiple retail
channels where possible or directly from the manufacturer. A set of two new printers was
used for the testing of each brand to assure uniformity and accuracy of the test data
independent of a particular printer. When cartridges were only available through one
vendor, cartridge markings were examined to ensure lot variation.
The cartridge bays were inspected and wiped clean of any residual toner particles and/or
paper dust before any new cartridge was installed. Once a cartridge was installed, the
printer would perform a full calibration before printing.
Printer and driver settings were left at factory default. The print technician did not agitate
the toner cartridges once they were installed into the printer. Color toner cartridges were
printed to End-of-Life.
Normal office conditions of temperature (75掳 卤8掳) and relative humidity (50% 卤10%) were
maintained for the duration of the test. All toner and paper consumables were stabilized in
these same conditions for a minimum of 24 hours prior to use.
Standard 8 陆 x 11 office paper (24 lb, 95-96 brightness) from HP, Hammermill, and Xerox
was used in sequential order, one ream (500 pages) at a time.
Each test page was serialized and identified as to the printer used to print the page, the
toner cartridges installed in the printer, and a sequential page number. Duplicate page
numbers from each brand were avoided by assigning each printer a different page range.
One printer was assigned a page range of 1 through 100,000 while the other printer used
for the brand was assigned a page range of 100,001 through 200,000.
Five test pages (files C1, C2, C3, C4, and C5) were designed using popular application
packages (Word XP, Excel XP, Adobe Illustrator 7.0, Adobe Acrobat, and InDesign CS) to
achieve a balanced representation of printing samples.
The test pages were sent to each printer in sequential order. Print quality assessments of
the pages printed by each toner cartridge and printer combination were compared at regular
intervals throughout the life of the cartridges in the test.
Page 14 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
Reliability Comparison Study
End-of-Life was determined when the printer required replacement of a cartridge before it
would continue printing or when the print technician observed serious degradation of print
quality. A set of pages would then be printed to isolate the problem to a particular color
cartridge (see samples below). To ensure that End-of-Life was not prematurely assigned,
two full calibrations of the printer were performed before the cartridge was declared to be
End-of-Life.
Magenta Test Page Spilling of yellow toner Back side of same page
(benchmark) across page
Using a computerized random number generator, a 10% random sample of total pages
printed (more than 670,000 pages) was identified for inspection before the pages were
printed. A square was placed in the lower right corner of the page to identify the pages to
be inspected. Trained page inspectors inspected these pages for print quality and ranked
each of them using the scale described in Appendix 1. The sample size was selected using
statistical methods to achieve statistically significant results.
After the print quality inspection, a subset of the pages marked with a square was
measured for color fidelity and consistency. Using an X-Rite Model 939
Spectrodensitometer, 22 CIE L*a*b* measurements were taken from each ream of paper
printed to determine color consistency and fidelity of each brand of cartridge. In total, close
to 30,000 measurements were taken across all brands.
Page 15 of 15
Test results provided by QualityLogic.
Tests were performed under laboratory conditions and your results may vary.
|