Input Parameter Modeling

General Input Parameter Modeling | General Combinatorial Testing | IPM in Combinatorial Testing | Random Testing | Syntax-based Testing | Benchmark | Research Groups | Software Tools | My Experiments | Models

    Research Papers: General Input Parameter Modeling

  1. read T. J. Ostrand and M. J. Balcer.,, The Category-Partition Method for Specifying and Generating Functional Tests., Communications of the ACM, 31(6):676–686, June 1988.

  2. read Kruse, P.M.; Wegener, J. , Test Sequence Generation from Classification Trees., Information Systems and Technologies (CISTI), 2011 6th Iberian Conference, 2011,p. 1-4

  3. Cain, A; Chen, TY; Grant, D; Poon, PL; Tang, S-F; Tse, TH ,An automatic test data generation system based on the integrated classification-tree methodology , Software Engineering Research and Applications, 2004,v. 3026, p. 225-238

  4. Lu Luo , Software Testing Techniques, Institute for software research international Carnegie mellon university Pittsburgh, PA 15232 USA. 2002: 1-19.

  5. read Grindal, M., Handling Combinatorial Explosion in Software Testing. , Thesis Dissertation no 1073, Department of Computer and Information Science Linköpings universitet, 2006. ISBN 978-91-87515-74-9. ISSN 0345-7524

  6. read T. Chen, P.-L. Poon, S.-F. Tang, and T. Tse, On the Identification of Categories and Choices for Specification-based Test Case Generation., Information and Software Technology, 46(13):887–898, 2004.

  7. read T. Chen, S.-F. Tang, P.-L. Poon, and T. Tse., Identification of Categories and Choices in Activity Diagrams., In Proceedings of the Fifth International Conference on Quality Software (QSIC 2005) 19-20 September 2005, Melbourne, Australia, pages 55–63. IEEE Computer Society, September 2005.

  8. Y. K. Malaiya. , Getting the most out of black-box testing. , In Proceedings of the International Symposium on Software Reliability Engineering, (ISSRE’95), Toulouse, France, Oct, 1995, pages 86–95, Oct. 1995.

  9. H. Yin, Z. Lebne-Dengel, and Y. K. Malaiya. , Automatic Test Generation using Checkpoint Encoding and Antirandom Testing , Technical Report CS-97- 116, Colorado State University, 1997.

  10. Benattou,M., Bruel, J., Hameurlain, N.,, Generating Test Data from OCL Specification, in Proceedings of the ECOOP'2002 Workshop on Integration and Transformation of UML models (WITUML02), 2002.

  11. A. A. Omar, F. A. Mohammed, survey of software functional testing methods, April 1991, SIGSOFT Software Engineering Notes , Volume 16 Issue 2

  12. Myers, G., Software Reliability: Principles and Practices, John Wiley, NY 1976

  13. readGoodenough, J.B., Gerhart, S.L., Toward a theory of test data selection, IEEE trans. softw. Eng. SE-2, June 1975, 156-173.

  14. Debra J. Richardson, Lori A. Clarke, A partition analysis method to increase program reliability, Proceedings of the 5th international conference on Software engineering, p.244-253, March 09-12, 1981.

  15. readElaine J. Weyuker And Thomas J. Ostrand,Theories of Program Testing and the Application of Revealing Subdomains , IEEE Transactions On Software Engineering, Vol. Se-6, No. 3, May 1980.

  16. T.Y. Chen, Pak-Lok Poon, T.H. Tse,A Choice Relation Framework for Supporting Category-Partition Test Case Generation, IEEE Transactions On Software Engineering, Vol. 29, No. 7, July 2003.

  17. Tang, Sau F., Phd Thesis: Identifying categories and choices for software testing based on informal specifications, A thesis submitted in total fulfillment of the requirements of the degree of Doctor of Philosophy, Swinburne University of Technology, 2009.

  18. Phil Stocks, David Carrington,A Framework for Specification-Based Testing, IEEE Transactions On Software Engineering, Vol. 22, No. 11, November 1996

  19. readAmit Paradkar, K.C. Tai, M.A. Vouk, Specification-based testing using cause-effect graphs, Annals of Software Engineering 4 (1997) 133–157.

  20. readMAN F. LAU, YUEN T. YU, An extended fault class hierarchy for specification-based testing, ACM Transactions on Software Engineering and Methodology (TOSEM), v.14 n.3, p.247-276, July 2005.

  21. readPraveen Ranjan Srivastava, Parshad Patel, Siddharth Chatrola, Cause effect graph to decision table generation, SIGSOFT Software Engineering Notes, March 2009, Volume 34, Number 2.

  22. readGutiérrez, J. J., Escalona M. J., Mejías M., Torres, J., Generation of test cases from functional requirements. A survey, 4th Workshop on System Testing and Validation. 2006. Germany.

  23. Paul Ammann and Jeff Offutt,Using Formal Methods To Derive Test Frames in Category-Partition Testing, Ninth Annual Conference on Computer Assurance (COMPASS 94), IEEE Computer Society Press pages 69-80, Gaithersburg, Maryland, June 1994.

  24. M. Balcer, W. Hasling, T. Ostrand, Automatic generation of test scripts from formal test specifications, SIGSOFT Software Engineering Notes Volume 14 Issue 8, December 1989.

  25. Boris Beizer, Software Testing Techniques. Van Nostrand Reinhold, NY, 2nd edition, 1990. ISBN 0-442-20672-0

  26. Balakrishnan Ramadoss, Paramasivam Prema, An Approach for Merging Two Classification-Trees, Proceedings of the IEEE International Advance Computing Conference, 1829-1834, 2009.

  27. Chen, T.Y.; Poon, P.L., Classification-Hierarchy Table: A Methodology for Constructing the Classification Tree, Software Engineering Conference, Pages 93-104, Australian, 1996

  28. readGlenford J. Myers., The Art of Software Testing, John Wiley & Sons, NY, 1979.

  29. readPak-Lok Poon, Tsong Yueh Chen, T.H. Tse, Comparing between CHOC’LATE and classification tree , Reliable Software Technologies: Ada-Europe 2012

  30. Matt Archer , Test Case Design with Classification Trees (Sample Book Chapter), A WebPage posted Posted by Matt Archer on March 7, 2012

  31. readGary E., B.Math., Requirements Based Testing Cause-Effect Graphing , Software Testing Services, 2005-2010 (Canadian Software Testing Board)

  32. Chen, T. Y., Poon, P.-L., Tang, S.-F., Tse, T. H., a DividE-and-conquer methodology for identifying categorieS, choiceS, and choicE Relations for Test case generation , IEEE Transactions on Software Engineering (2011) doi: 10.1109/TSE.2011.69

  33. Pak-Lok Poon, Sau-Fun Tang, T.H. Tse, and T.Y. Chen, CHOCLAT: a CHOiCe reLATion framEwork for specification-based testing , , Communications of the ACM 53 (4): 113-118 (2010)

    Research Papers: General Combinatorial Testing

  1. Jazek Czerwonka., Pairwise Testing in Real World (Practical Extensions to Test Case Generators)., practical extensions to test case generators, 2006.

  2. Lei, Y., Carver, R. H., Kacker, R., and Kung, D. C. , A combinatorial testing strategy for concurrent programs, . Softw. Test., Verif. Reliab. 2007, 17, 4, 207–225.

  3. I. S. Dunietz, W. K. Ehrlich, B. D. Szablak, C. L. Mallows, A. Iannino, Applying design of experiments to software testing , Proceeding ICSE '97 Proceedings of the 19th international conference on Software engineering

  4. Renée C. Bryce , Charles J. Colbourn,,Test prioritization for pairwise interaction coverage , Proceedings of the 1st international workshop on Advances in model-based testing, p.1-7, May 15-21, 2005.

  5. Renée C. Bryce , Atif M. Memon,Test Suite Prioritization by Interaction Coverage , Workshop on Domain specific approaches to software test automation: in conjunction with the 6th ESEC/FSE joint meeting, p.1-7, September 04-04, 2007.

  6. Renée C. Bryce, Yu Lei, D. Richard Kuhn, Raghu Kacker ,Chapter 14 - Combinatorial Testing , 2010 Combinatorial testing. En Software Engineering and Productivity Technologies, M. Ramachandran and R.A.d. Carvalho (eds.). Idea Group Publishing. p. 196-208

  7. Lei Xu, Baowen Xu, Changhai Nie, Huowang Chen, Hongji Yang ,A Browser Compatibility Testing Method Based on Combinatorial Testing , In Proc. of the Int. Conference on Web Engineering (ICWE), 2003, pp. 310--313.

  8. Shi, L., Nie, C., Xu, B. , A Software Debugging Method Based on Pairwise Testing , In Proceedings of the International Conference on Computational Science.,200 Springer, 1088--1091.

  9. Dean Hoskins , Renée C. Turban , Charles J. Colbourn , Experimental designs in software engineering: d-optimal designs and covering arrays , Proceedings of the 2004 ACM workshop on Interdisciplinary software engineering research, November 05-05, 2004.

  10. D.R. Kuhn and M.J. Reilly, An Investigation of the Applicability of Design of Experiments to Software Testing , Proc. 27th NASA/IEEE Software Eng.Workshop, Dec. 2002.

  11. Cohen, D.M.; Dalal, S.R.; Fredman, M.L.; Patton, G.C., The AETG System An Approach to Testing Based on Combinatorial Design, IEEE Transactions on Software Engineering, Volume 23, Issue 7,Pages 437-444, July 1997.

  12. Mats Grindal, Evaluation of Combination Strategies for Practical Testing, PhD thesis, Skovde University, Sweden, 2007.

  13. Mats Grindal, Birgitta Lindström, Jeff Offutt, and Sten F. Andler., An Evaluation of Combination Testing Strategies, Empirical Software Engineering, 11(4):583-611, December 2006.

  14. , ,

  15. , ,

    Research Papers: IPM in Combinatorial Testing

  1. Grindal, M. and Offutt, J. , Input Parameter Modeling For Combination Strategies., Proceedings of the IASTED International Conference on Software Engineering (SE2007), Innsbruck, Austria, 13-15 Feb 2007, pages 255-260

  2. Grindal, M., Offutt, J. and Mellin, J. , Managing Conflicts when Using Combination Strategies to Test Software., Proceedings of the 18th Australian Conference on Software Engineering (ASWEC2007), Melbourne, Australia, 10-13 April 2007.

  3. Grindal, M, Offutt, J, and Andler, S. F. , Combination Testing Strategies: (A) Survey., publisher Wiley, Software Testing, Verification, and Reliability,2004, volume 15, number 2, pp. 167-199.

  4. D. M. Cohen, S. R. Dalal, J. Parelius, and G. C. Patton., The Combinatorial Design Approach to Automatic Test Generation, IEEE Software, 13(5):83–89, September 1996.

  5. Wenhua Wang, Sreedevi Sampath, Yu Lei, Raghu Kacker. , An Interaction-Based Test Sequence Generation Approach for Testing Web , IEEE International Conference on High Assurance Systems Engineerng, December 2008.

  6. read Kuhn R, Wallace D, Gallo A., Software fault interactions and implications for software testing , IEEE Transactions on Software Engineering 2004; 30(6):418–421.

  7. C. Nie and H. Leung. , A survey of combinatorial testing, ACM Computing Surveys (CSUR), 43:11:1–11:29, 2011

  8. Krishnan, R.,Krishna, S. M., Nandhan, P. S. , Combinatorial testing Learnings from our experience , Sigsoft Softw. Engin., 2006, Notes 32, 3, 1–8.

  9. Burr, K. and Young, W. , Combinatorial Test TechniquesTable-based AutomationTest Generation And Code Coverage , In Proceedings Of The International Conference On Software Testing Analysis And Review. 503–513, October 1998.

  10. Shams, M., Krishnamurthy, D., Far, B., A model-based approach for testing the performance of web applications , Proceedings of the Third International Workshop on Software Quality Assurance (SOQUA06)

  11. Mandl, R. Corporation, C., Orthogonal Latin squares: an application of experiment design to compiler testing , Magazine Communications of the ACM CACM Homepage archive Volume 28 Issue 10, Oct. 1985

  12. White, L. Almezen, H., Generating test cases for GUI responsibilities using complete interaction sequences, Software Reliability Engineering, 2000. ISSRE 2000. Proceedings. 11th International Symposium, 110-121

  13. D. Richard Kuhn, Raghu N. Kacker, Yu Lei , PRACTICAL COMBINATORIAL TESTING, National Institute of Standards and Technology, 2010.

  14. Burroughs, K., Jain, A., and Erickson, Improved Quality Of Protocol Testing Through Techniques Of Experimental Design. , In Proceedings of the IEEE International Conference 1994, Vol. 2. 745–752.

  15. readItai Segall, Rachel Tzoref-Brill, Aviad Zlotnick, Simplified Modeling of Combinatorial Test Spaces, To Appear, 1st International Workshop on Combinatorial Testing (in conjunction with ICST'12), 2012.

  16. Vilkomir, S.A.; Swain, W.T.; Poore, J.H.; Software Input Space Modeling with Constraints among Parameters, Computer Software and Applications Conference, 2009.

  17. readItai Segall, Rachel Tzoref-Brill, Aviad Zlotnick, Using binary decision diagrams for combinatorial test design, Proceedings of the 2011 International Symposium on Software Testing and Analysis.

  18. C. Lott, A. Jain, S. Dalal, Modeling Requirements for Combinatorial Software Testing, SIGSOFT Softw. Eng. Notes,30:1-7, 2005.

  19. readD.R. Wallace and D.R. Kuhn, Failure Modes in Medical Device Software:An Analysis of 15 Years of Recall Data, Int’l J. Reliability, Quality and Safety Eng., vol. 8, no. 4, 2001.

  20. J. Pan, The Dimensionality of Failures—A Fault Model for Characterizing Software Robustness, Proc. Int’l Symp. Fault-Tolerant Computing, June 1999.

  21. Itai Segall, Rachel Tzoref-Brill, Aviad Zlotnick, Common Patterns in Combinatorial Models , 1st International Workshop on Combinatorial Testing (in conjunction with ICST'12), 2012.

  22. Itai Segall, Rachel Tzoref-Brill, Interactive Refinement of Combinatorial Test Plans, International Conference on Software Engineering (ICSE'12), 2012.

  23. Elke Salecker, Sabine Glesner, Combinatorial Interaction Testing for Test Selection in Grammar-Based Testing, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation slides

    Research Papers: Random Testing

  1. Richard Kuhn, Raghu Kacker, Yu Lei., Combinatorial and Random Testing Effectiveness for a Grid Computer Simulator presented at the Mod Sim World 2009, Virginia, USA

  2. P. J. Schroeder, P. Bolaki, and V. Gopu, Comparing the fault detection effectiveness of n-way and random test suites, In Proceedings of the IEEE International Symposium on Empirical Software Engineering, pages 49–59, 2004.

  3. Mario Brčić and Damir Kalpić Combinatorial testing in software projects Jubilee 35th International Convention Proceedings / Biljanović, Petar (ur.). - Rijeka : Croatian Society for Information and Communication Technology, Electronics and Microelectronics - MIPRO , 2012. 1832-1837

  4. Pretschner, A.; Mouelhi, T.; Le Traon, Y., Model-Based Tests for Access Control Policies, International Conference on Software Testing, Verification, and Validation pp. 338-347, 2008.

    Research Papers: Syntax-based Testing

  1. read E. Soechting, K. Dobolyi, and W. Weimer., Syntactic Regression Testing for Tree-Structured Output. In Proc. Int. Symp. on Web Systems Evolution (WSE'09). IEEE Computer Society, 2009

  2. Zhiqiang Lin , Xiangyu Zhang,, Input Syntactic Structure. Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, November 09-14, 2008, Atlanta, Georgia


  1. IBM CTD Benchmark            
  2. SIR Benchmark            
  3. IBM. Combinatorial Test Services            
  4. Traceability Benchmark (AquaBench)            
  5. RFC index            

    Research Groups:

  1. IBM Research - Haifa             IBM

  2. NIST                          

  3. Open Research Datasets in Software Engineering                          

    Software Tools

Tools Platform Property Title Description
TESSY N/A N/A Hitex develoment tools A Unit Testing tool which uses the classification tree method
FoCuS IBM Test Planning and Coverage Analysis, it allows for user definition of a model, and then it performs advanced analyses that produce different orthogonal views of the model states that help locate omission problems and design mistakes. Also provides extensive views and analyses for code coverage.
CTE XL The classification-tree editor (CTE) is a graphical editor supporting the application of the classification-tree method. CTM and CTE are widely used in industrial practice.
combinatorial testing tools