Quality Assurance Project Plan (QAPP)
Table of Contents
- Elements: Analytical Methods Common to Most QAPPs Prepared for Equivalency Recommendations
- Design - Goals and Objectives, Scale, and Scope
- Elements: Quality Assurance & Quality Control Measures
- Design - Pointers on Select Details
In reviewing Process to Further Reduce Pathogens and Process to Significantly Reduce Pathogens equivalency applications, the Pathogen Equivalency Committee (PEC) will verify that the results submitted in support of a process are statistically significant and were acquired taking into account appropriate quality assurance and quality control measures. (See Tip box below.) In some cases, the PEC may conduct on-site reviews.
A quality assurance project plan (QAPP) should be developed before beginning testing so that the desired quality in sample collection, laboratory analysis, data validation and reporting, and documentation and record keeping is achieved and maintained. A QAPP is a written document that provides a blueprint for the entire project and each specific task to ensure that the project produces reliable data that can be used to meet the project's overall objectives and goals.
Applicants are now required to prepare and submit a quality assurance project plan for the PEC to review prior to conducting any research in support of their application for equivalency. Very few exceptions to this requirement will be granted. PEC review and approval of an applicant’s project plan prior to data collection will save time and money in the long run by ensuring that the proper data is collected in an appropriate manner and unnecessary data collection is eliminated.
The information required in the application for equivalency recommendation is largely drawn from the QAPP requirements. The extensive overlap between the QAPP and the application is evident in the combined Completeness Checklist below that the PEC uses to evaluate both documents. For this reason, careful preparation of a QAPP will serve as a good head start on the application itself. To assist the applicant in developing such a plan, QAPP Guidelines for Applied Research Projects are provided below. This guidance document contains information on:
- requirements for the project description and objectives
- project organization
- experimental approach
- sampling procedures
- testing and measurement protocols
- Quality Assurance/Quality Control checks
- data reporting
- data reduction
- data validation
- assessments and
Some basic information on sampling procedures and analytical methods which may be useful in the development of a QAPP are summarized on this page and discussed in greater detail in:
US EPA (2003) "Environmental Regulations and Technology: Control of Pathogens and Vector Attraction in Sewage Sludge." Chapter 9 (PDF) (11 pp, 601K) Publication No. EPA/625/R-92/013.
A list of quality management tools, including references and training to assist with the development of a QAPP can be found on the EPA's Quality System website.
The Uniform Federal Policy for Quality Assurance Project Plans (EPA/505/B-04/900A) is another document you may wish to reference. This very thorough document is the result of collaboration between EPA, DoD, and DOE to standardize QAPP requirements and definitions. In addition to consulting these publications, applicants are encouraged to contact a Quality Assurance/Quality Control expert, statistician, or the PEC as they develop their testing plan, to discuss the quality assurance project plan and proposed sampling techniques.
Fecal Coliform (either EPA method is preferred)
- EPA Method 1680 (PDF) (51 pp, 1.1MB): Fecal Coliforms in Sewage Sludge (Biosolids) by Multiple-Tube Fermentation using Laurel-Tryptose Broth (LTB) and EC Medium
- EPA Method 1681 (PDF) (50 pp, 889K): Fecal Coliforms in Sewage Sludge (Biosolids) by Multiple-Tube Fermentation using A-1 Medium
- Standard Methods for the Examination of Water and Wastewater Methods (18th ed.) 9221 E: Multiple-Tube Fermentation Technique Fecal Coliform Procedure, used in conjunction with Control of Pathogens and Vector Attraction in Sewage Sludge, Appendix F: Sample Preparation Fecal Coliforms in Sewage Sludge (Biosolids) by Multiple-Tube Fermentation using A-1 Medium (PDF) (4 pp, 366K) (EPA/625/R-92/013) Revised July 2003
- Standard Methods for the Examination of Water and Wastewater Methods (18th ed.) 9222 D: Membrane Filter Technique Fecal Coliform Procedure, used in conjunction with Control of Pathogens and Vector Attraction in Sewage Sludge, Appendix F: Sample Preparation Fecal Coliforms in Sewage Sludge (Biosolids) by Multiple-Tube Fermentation using A-1 Medium (PDF) (4 pp, 366K) (EPA/625/R-92/013) Revised July 2003 (acceptable for PSRP equivalencies only)
- EPAMethod 1682 (PDF) (48 pp, 1MB): Salmonella in SewageSludge (Biosolids) by Modified Semisolid Rappaport-Vassiliadis (MSRV)Medium
- EPA/625/R-92/013, AppendixG (PDF) (9 pp, 425 KB): Kenner and Clark (1974) Analytical Method for Salmonella sp. Bacteria, used in conjunction with Control of Pathogens and Vector Attraction in Sewage Sludge, Appendix F: Sample Preparation for Fecal Coliform Tests and Salmonella sp. Analysis (PDF) (4 pp, 366K) (EPA/625/R-92/013) Revised July 2003
- Control of Pathogens and Vector Attraction in Sewage Sludge, Appendix H: Method for the Recovery and Assay of Total Culturable Viruses from Sludge (PDF) (16 pp, 355 KB) (EPA/625/R-92/013) Revised July 2003
Viable Helminth Ova
- Control of Pathogens and Vector Attraction in Sewage Sludge, Appendix I: Test Method for Detecting, Enumerating, and Determining the Viability of Ascaris Ova in Sludge (PDF) (7 pp, 510K) (EPA/625/R-92/013) Revised July 2003
Percent Total and Volatile Solids (either method is acceptable)
- EPA Method 1684: Total, Fixed, and Volatile Solids in Water, Solids, and Biosolids
- Standard Methods for the Examination of Water and Wastewater Methods 2540 B & E: Total Solids Dried at 103 - 105°C and Fixed and Volatile Solids Ignited at 500°C or Method 2540 G: Total, Fixed, and Volatile Solids in Solid and Semisolid Samples
Optional and Surrogate Indicator Organisms
Specific methods for the analysis of optional and surrogate indicator organisms are not mandatory unlike the organisms used for microbial compliance monitoring of biosolids whose methods (as listed above) are specified by 40 CFR 503.8. However, acceptable protocols for analysis of microbial indicator organisms and specific pathogenic microorganisms have been developed for:
- Foods and
- Other matrixes which have been incorporated into compendiums of industry standard assays or defined as Agency approved methods.
Specifically, the references listed below identify methods which may be useful for analysis of biosolids. Caution should be used when selecting and using methods for alternate or surrogate indicator organisms since none of these methods have been subject to multi-laboratory validation studies for sewage sludge or biosolids.
- American Public Health Association, Standard Methods for the Examination of Water and Wastewater 21st Edition. 2005. Washington D.C.
- American Public Health Association, Compendium of Methods for the Microbiological Examination of Foods 4th Edition. 2001. Washington D.C.
- U.S. Food and Drug Administration, Bacteriological Analytical Manual 8th Edition. 1998. Washington D.C.
- U.S. Environmental Protection Agency, EPA Microbiology website. 2007.
Goals and Objectives
The goals and objectives of a QAPP prepared for the purposes of supporting an equivalency application will have the same standard components. The goal of such a QAPP will be to support the equivalency of the process in question to a process to further (or significantly) reduce pathogens on a site-specific or national basis.
The objectives will vary depending on the type of equivalency because the criteria for verifying efficiency differ between a process to significantly reduce pathogens and a process to further reduce pathogens. (Equivalency Criteria webpage.) However, regardless of the equivalency type, the data should be defensible in demonstrating that the process is consistently capable of pathogen reduction on par with accepted processes.
Typically, to receive an equivalency recommendation, laboratory work is performed to establish the boundary conditions of all key process variables, and then pilot or full-scale testing is performed to demonstrate successful scale-up. The required elements of a QAPP can be quite different depending on scale. If laboratory-scale testing is to be used, the applicant may find it easier to divide their research into two phases and carry out the work under separate QAPPs, one for the laboratory-scale work and one for the scale-up work.
Planning a pilot-scale study includes some special considerations if one of its goals is to gather data for scaling up the process to a plant scale. The pilot unit should be truly representative of a full-scale operation. (Definitions of Scale.) The conditions of the pilot-scale operation should be no more severe than those expected of the full-scale operation. These conditions will likely include for example:
- degree of mixing,
- nature of the flow (batch vs. flow though units and degree of short-circuiting in flow through units),
- vessel sizing, and
- proportions of the chemicals used.
Any substantial departure in process parameters between the pilot-scale and the full-scale systems that has the potential to reduce the effectiveness of the process will invalidate any approvals given and will require a retest at the new condition.
As discussed in the Basic Information web page, whether the goal is a site-specific or a national equivalency will also play a role in the overall QAPP design. For site-specific equivalencies work only needs to be performed on sludge collected from one location. For a national equivalency, however, the work must be repeated with significantly varying sludges. This would entail a combination of laboratory studies on a wide variety of sludges followed by scale-up testing using at least one sludge/location. Or if preferred, a mobile pilot-unit could be constructed and used for all testing, eliminating the need for laboratory studies.
Note that the pilot unit must be a true pilot-scale of the final full-scale system for the equivalency recommendation to apply to the full-scale and not just the pilot scale. (See the pilot-scale definition for more discussion on what is considered a true pilot-scale.)
- Quality Assurance Measures refer to protocols or activities undertaken to assure the reliability of the data collected. These measures generally apply widely to the project as a whole. Examples of quality assurance measures include, but are not limited to the following:
- Holding times for microbial samples are generally 24 hours or less. A provision for checking holding times and consequences of holding time exceedances should be included.
- Sample representativeness with respect to sampling and handling procedures can be assured through the use of duplicate sampling. An acceptable range of relative percent difference between a sample and its duplicate (typically 20%) should be set. Data falling outside this range is invalid.
- Sample representativeness with respect to sample processing and analysis can be assured through replicate analysis. An acceptable range of relative standard deviation among replicate analyzes (typically 10%) should be set. Data falling outside this range is invalid.
- Calibration and maintenance procedures, schedules, and standards (if applicable) must be specified for all equipment used through the study. For example, temperatures of refrigerators and incubators used should be verified with independent thermometers on a regular basis. Calibration of pH probes should be performed using appropriate standard solution ranges, etc.
- Quality Control Measures refer to actions included to assure that defined standards are met in the analysis of data. These measures are generally analyte or method specific and are often defined within the method procedure itself. Not all types of controls are necessary for every analyte. Examples of quality control measures that should be incorporated in a QAPP designed to support process equivalency include, but are not limited to the following:
- Method blanks to ensure the workspace, handling procedures, and reagents are free from contamination. For example, processing a sample of reagent-grade water along with normal samples when measuring percent total solids. This method blank may not be zero, but should be insignificant compared to the actual samples (e.g., % solids of the blank should be less than 10% of the lowest measured sample) or the entire data set will be invalid. Some method blanks may be a simple positive or negative. For example, incubation of one MPN tube without inoculation (media control) and one MPN tube after inoculation with pure dilution water when measuring fecal coliform.
- Positive and negative controls establish that the method is working as designed. For these controls, something known to produce the appropriate effect is to be added. For example, inoculating MPN tubes with pure cultures of E. coli and Enterobacter spp. would be a positive and negative control, respectively, for fecal coliform measurement.
- Matrix Spikes are necessary for methods that are known to have low or inconsistent percent recoveries. This includes helminth (Ascaris) ova and enteric viruses, which can have percent recoveries as low as 10-30%. In matrix spikes, known quantities of the analyte are added to the sample. After subtracting out the background level naturally present in the unspiked sample, a percent recovery can be calculated by dividing the measured value by the known spiked value. The matrix will affect the percent recovery so this test must be performed for the untreated and the finished product.
Proper project design and sampling techniques are of utmost importance to producing a successful QAPP. Though not all-inclusive, some important points to consider when planning your QAPP include:
- Accepted, state-of-the-art techniques for sampling and analyzing sludge should be used to ensure the quality of the data.
- The choice of sampling device should be appropriate for the physical characteristics of the sludge (viscosity and solids content).
- Effort must be made to minimize the possibility of sample contamination.
- The samples should be representative of the random and cyclic variation in sludge characteristics that occur during treatment. Representative samples can be obtained by compiling composite samples over volume, by ensuring that each grab sample or aliquot of a composite sample is as representative as possible of the total stream flow passing the sampling point, by establishing an appropriate frequency of sampling that accounts for variation, and by taking an appropriate number of samples to account for variation.
- A pair of input and output samples of non-batch systems can be drawn simultaneously. However, to ensure that measurements are independent, samples should not be taken on successive days. At least one estimated sludge retention time should separate each successive pair of input and output samples.
- If ambient conditions affect sludge microbial characteristics, sludge should be sampled after treatment under the least favorable conditions.
- Sampling, packaging, and shipping procedures should not alter the sludge character or quality.
- Laboratories providing analytical services should be experienced in the analysis of municipal sludges and biosolids and should be able to demonstrate compliance with analytical quality assurance protocols.
- Field verification and documentation by independent or third-party investigators is desirable.
Below are some resources to assist in the development of a successful and useful quality assurance project plan. Two example QAPPs and mock reviews of these QAPPs using the Completeness Checklist are provided. Although neither QAPP pertains to biosolids or, more specifically, the planning of a project to demonstrate Process to Further Reduce Pathogens or Process to Significantly Reduce Pathogens equivalency, the examples and mock reviews do illustrate the type of information that is required in a well-written QAPP.
QAPP Guidelines for Applied Research Projects (PDF) (5 pp, 110K) | 508-Compliant Version
An annotated outline providing specific information on what to include in a well designed quality assurance project plan.
Completeness Checklist (DOC) (8 pp, 263K)
This checklist is used by the PEC to review submitted QAPPs and equivalency applications. It is provided to help applicants double-check that all required and applicable elements have been addressed in their QAPP/equivalency application before submittal.
Example QAPP - Biofilm Growth (DOC) (16 pp, 220K)
This example QAPP is entitled Growth rate of Biofilm Organisms in a Distribution System.
QAPP Checklist for Biofilm Growth (DOC) (8 pp, 251K)
A Completeness Checklist was filled out for the biofilm growth QAPP in a mock review showing the strengths and weaknesses.
Example QAPP - Particulate Nutrients (DOC) (19 pp, 236K)
This example QAPP is entitled Analysis of Particulate Bound Nutrients in Stormwater.
QAPP Checklist for Particulate Nutrients (DOC) (9 pp, 252K)
A Completeness Checklist was filled out for the particulate nutrients QAPP in a mock review showing the strengths and weaknesses.
EPA's Quality System Web site
Additional resources to assist in the development of a quality assurance project plan can be found on this Web site.
Uniform Federal Policy for QAAPs
This document is a useful reference that provides detailed definitions of common QAPP elements.