NASA's TRLs http://www.hq.nasa.gov/office/codeq/trl/trl.pdf (5 page white paper) http://esto.nasa.gov/files/TRL_definitions.pdf (1 page definitions) - Covers technology more than software, which generally seems to be hardware; tends to use general descriptions rather than quantitative measures; 1-9 scale NASA TRLs Applied to Software: http://isd.gsfc.nasa.gov/Technology/TRL/TRL.ppt - Modification of standard TRLs to apply to software; tends to use general descriptions rather than quantitative measures; 1-9 scale Technology Readiness Assessments for IT and IT-Enabled Systems http://www.stsc.hill.af.mil/crosstalk/2005/05/0505Gold.pdf - Based on DoD work; TRA is a process of IDing critical elements, evaluating their TRLs, and planning their maturation; TRLs here are similar to NASA software TRLs; tends to use general descriptions rather than quantitative measures; 1-9 scale OPEN Process Framework's Technology Readiness Assessment http://www.opfro.org/Components/WorkProducts/ArchitectureSet/TechnologyReadinessAssessment/TechnologyReadinessAssessment.html - Also a process where critical elements are IDed before assessing their TRLs; includes definitions for both new and reused critical technology; reused items start half way up the scale; lower TRLs for new items are close to NASA TRLs, but higher ones seem to spread out a bit more (e.g., 7, 8, and 9 are passing alpha, beta, and operational tests); 1-10 scale Using the TRLs Scale to Support Technology Management in the DoD's ATD/STO Environments http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02sr027.pdf - Has two tables in appendix; first based on DoD work and similar to NASA software TRLs, second is Army draft and gives separate descriptions for hardware and software (hardware scale matches first table, software scale looks like it's a bit more detailed than the others); document includes a section on a TRL Calculator tool; both TRL scales here tend to use general descriptions rather than quantitative measures; 1-9 scale Alternative to TRLs for Non-Developmental Item (NDI) Software http://csdl2.computer.org/comp/proceedings/hicss/2005/2268/09/22680315a.pdf (8 page conference proceedings) http://www.sei.cmu.edu/pub/documents/04.reports/pdf/04tr013.pdf (full technical report) - Notes that TRLs have problems when applied to NDI software like COTS, GOTS, OSS, and suggests an alternative system for rating software readiness; uses a set of attributes: requirements, environmental fidelity, product criticality, product availability, product maturity; an example shows how an assessment would be made using the Analytic Hierarchy Process; evaluation levels in attributes are still more descriptive than quantitative, and this seems to be used more for comparing products to determine which is more ready for a particular use, as one product doesn't get one single numerical rating (it has as many ratings as there are attributes); conclusions note that this method has not been applied to an actual system development yet ImpACT: Alternative to TRLs for COTS Software http://www.iccbss.org/2004/proceedings/ImpACT.pdf - By same author as the NDI papers above, and with a similar alternative; attributes here are importance, availability, capability, and timeframe; each gets a value 0-5; an example evaluation is provided and is much like the one in the NDI paper; both papers also note that importance of different attributes can vary over time and with the maturity of the technology; the evaluation levels in the attributes are still more descriptive than quantitative, and this seems to be used more for comparing products to determine which is more ready for a particular use, as one product doesn't get one single numerical rating Business Readiness Rating for Open Source http://www.openbrr.org/docs/BRR_whitepaper_2005RFC1.pdf - Proposes an open and standard model for assessing software, and the model should be complete, simple, adaptable, and consistent; software's Business Readiness Rating is score from 1-5 by evaluating it in different areas including functionality, usability, quality, security, performance, scalability, architecture, support, documentation, adoption, community, and professionalism; appendix contains more details including representative metrics and their scoring, which are generally quantitative (e.g., time for vanilla installation is based on the time it takes, security and quality factors are suggested to be measured in terms of numbers of bugs, patches, etc.); these could certainly be modified, but it's the only one of these that has quantitative measures instead of just descriptive measures; describes the method for making an assessment and offers some best practice suggestions for the different stages TRL Corollaries for Practice-Based Technologies http://www.sei.cmu.edu/products/events/acquisition/2003-presentations/graettinger.pdf - For practices, processes, methods, approaches, and frameworks; uses the same short definition of TRLs (looks like DoD, but NASA may be the same), but uses a more detailed, technology-independent description; may be useful if we plan to store information like this in the RES Research & Development Degree of Difficulty (R&D^3) white paper http://www.hq.nasa.gov/office/codeq/trl/r&d3.pdf - Proposed as an additional measure to TRLs; a measure of the level of difficulty expected in maturing technology; probably not very relevant for our needs A TRL Calculator has been created by William Nolte (USAF), available at https://acc.dau.mil/simplify/ev_en.php?ID=8796_201&ID2=DO_TOPIC However, an account on the site is required to access it. A description of the program can be found at: http://www.dtic.mil/ndia/2003systems/nolte.ppt