s519 logo S519 Evaluation of Information Systems

Course Readings

Home | About | Course Readings | Syllabus | Handouts | Assignment


Week 1: Introduction of the course and cases.

Hayes, M., Murphy, C., Khirallah, D. R., Hulmes, G. V., & Ricadela, A. (2002, May 20). Quality first. Information Week, 889, 38-47.

Skim: ISWORLD site on Information Systems Effectiveness


Week 2:Understanding evaluation

Davidson, E. J. (2005). Chapters 1-5.

Ward, J. A. (1996). Measurement management: what you measure is what you get. Information Systems Management, 13(1), 59-61. [E-Reserve]

Week 3: Cases & Excel

Salkind, N. J. (2007). Chapters 1, 1a, & 1b.

Read cases:

Week 4: Identifying issues & formulating questions

Davidson, E. J. (2005). Chapter 6.

DeLone, W. H., & McLean, E. R. (1992). Information systems success: the quest for the dependent variable. Information Systems Research, 3(1), 60-95.[E-Reserve]

DeLone, W. H., & McLean E. R. (2002). Information systems success revisited. Proceedings of the 35th Annual Hawaii International Conference on System Science. [Online]

Salkind, N. J. (2007). 3rd ed. Chapter 6. [E-Reserve]

Optional Reading:

Chambliss, D. F. & Schutt, R. K. (2006). Making sense of the social world: Methods of investigation (2nd ed.). Thousand Oaks, CA: Sage. Chapter 3: Conceptualization and measurement, p.51-79. Chapter 4: Sampling, p. 85-102. [E-Reserve]

DeLone, W. H., & McLean E. R. (2003). The DeLone and McLean Model of Information Systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. [E-Reserve]

Xie, H. (2006). Evaluation of digital libraries: Criteria and problems from users' perspectives. Library & Information Science Research, 28, 433-452. [E-Reserve]

Week 5: Reporting of evaluation

Davidson, E. J. (2005). Chapter 10 [skim].

Salkid, N. J. (2007). Chapters 6, 7, & 8.

Babbie, E. (2009). The practice of social research (11th ed.). Belmont, CA: Wadsworth. Chapter 16. [E-Reserve]


Week 6: System/software quality; Information/data quality

System/software quality

Aladwani, A. M., & Palvia, P. C. (2002). Developing and validating an instrument for measuring user-perceived Web quality. Information & Management, 39(6), 467-476. [E-Reserve]

Information/data quality

Agosta, L. (2005). Trends in data quality. DM Review, 15(2), 34-35. [Online]

Chae, M., Kim, J., Kim, H., & Ryu, H. (2002). Information quality for mobile internet services: A theoretical model with empirical validation. Electronic Markets, 12(1), 38-46. [E-Reserve]

Strong, D. M., Lee, Y. W., & Wang, R. Y. (1997). 10 potholes in the road to information quality. Computer, 30(8), 38-46. [E-Reserve]

Stvilia, B., Twidale, M. B., Gasser, L., Smith, L. C. (2005). Information quality discussions in Wikipedia. Technical Report ISRN UIUCLIS--2005/2+CSCW. [Online]

Optional reading:

Belanger, F., Schaupp, W. F. L. C., Krishen, A., Everhart, J., Poteet, D., & Nakamoto, K. (2006). Web site success metrics: Addressing the duality of goals. Communications of the ACM, 49(12), 114-116. [E-Reserve]

Kim, Y. J., Rajiv, K., & Lawrence, S. G. (2005). From DQ to EQ: Understanding data quality in the context of e-business systems. Communications of the ACM, 48(1), 75-81. [E-Reserve]

Mankoff, J., Fait, H., & Tran, T. (2005). Is your web page accessible?: A comparative study of methods for assessing web page accessibility for the blind. Proceedings of the SIGCHI conference on Human Factors in computing systems, 41-50. [E-Reserve]

Murayama, Y., Hikage, N., Hauser, C., Chakraborty, B., & Segawa, N. (2006). An Anshin model for the evaluation of the sense of security. Proceedings of the 39th Hawaii International Conference on System Sciences. [E-Reserve]

Pipino, L. L., Lee, Y. W., & Wang, R. Y. (2002). Data quality assessment. Communications of the ACM, 45(4), 211-218. [Online]


Week 7: Usability/performance; attitudes/satisfaction


Désilets, A., Paquet, S., & Vinson, N.G. (2005). Are wikis usable? Proceedings of the 2005 International Symposium on Wikis, 3-15. [E-Reserve]

Palmer, J. W. (2002). Web site usability, design, and performance metrics. Information Systems Research, 13(2), 151-167. [E-Reserve]

Rosenbaum, S., Rohn, J. A., & Humburgh, J. (2000). A toolkit for strategic usability: results from workshops, panels, and surveys. Proceedings of the SIGCHI conference on Human factors in computing systems, 337-344. [E-Reserve]

User satisfaction

Nevo, D., & Wade, M. R. (2007). How to avoid disappointment by design. Communications of the ACM, 50(4), 43-48. [E-Reserve]

Ong, C. S., & Lai, J.-Y. (2004). Developing an instrument for measuring user satisfaction with knowledge management systems. Proceedings of the 37th Annual Hawaii International Conference on System Sciences, 262-271. [E-Reserve]

Optional reading:

Doll, W., Deng, X., Raghunathan, T. S., & Torkzadeh, G., et al. (2004). The measuring and measurement of user satisfaction: A multigroup invariance analysis of the end-user computing satisfaction instrument. Journal of Management Information Systems, 21(1), 227-262. [skim the method and results sections] [E-Reserve]

Kerchner, M. (2006). A dynamic methodology for improving the search experience. In Information Technology & Libraries, 25(2), 78-87. [E-Reserve]

Ozok, A. A., & Salvendy, G. (2004). Twenty guidelines for the design of web-based interfaces with consistent language. Computers in Human Behavior, 20(2), 149-152. [E-Reserve]

Week 8: Implementation/use

Boudreau, M., & Seligman, L (2005). Quality of use of a complex technology: A learning-based model. Journal of Organizational and End User Computing, 27(4), 1-22. [E-Reserve]

Chen, J. (2007). Flow in games (and every thing else). Communications of the ACM, 50(4), 31-34. [E-Reserve]

Prisk, D. P., & Brooks, M. G. (2005). Hip high-tech purchases don't always work out as planned. Computers in Libraries, 25(10), 10-16. [E-Reserve]

Legris, P., Ingham, J., & Collerette, P. (2003). Why do people use information technology? A critical review of the technology accepted model. Information & Management, 40(3), 191-205. [E-Reserve]

Norguet, J., Zimanyi, E., & Steinberger, R. (2005). Improving web sites with web usage mining, web content mining, and semantic analysis. Proceedings of the 32nd International Conference on Current Trends in Theory and Practice of Computer Science. [Online]

Optional reading:

Benbasat, I., & Barki, H. (2007). Quo vadis TAM? Journal of the Association of for Information Systems, 8(4). [Online]

Chung, J., & Felix, T. (2004). Antecedents of perceived playfulness: An exploratory study on user acceptance of general information searching websites. Information & Management, 41(7), 869-882. [E-Reserve]

Karahanna, E., Agarwal, R., & Angst, C. M. (2006). Reconceptualizing compatibility beliefs in technology acceptance research. MIS Quarterly, 30(4), 781-804. [E-Reserve]

Majchrzak, A., Wagner, C., & Yates, D. (2006). Corporate wiki users: Results of a survey. Proceedings of the 2006 International Symposium on Wikis, 99-104. [E-Reserve]

Mahmood, M. A., Hall, L., & Swanberg, D. L. (2001). Factors affecting information technology usage: A meta-analysis of the empirical literature. Journal of Organizational Computing & Electronic Commerce, 11(2), 107-130. [E-Reserve]

Rubin, J. H. (2001). Introduction to log analysis techniques: Methods for evaluating networked services. In C. R. McClure & J. C. Bertot (Eds.), Evaluating networked information services: Techniques, policy and issues (pp. 197-212). Medford, NJ: Information Today Inc. [E-Reserve]

Venkatesh, V., & Viswanath, R. V. (2006). Web and wireless site usability: Understanding differences and modeling use. MIS Quarterly, 30(1), 181-205. [E-Reserve]

Week 9: Net effects

Corea, S. (2006). Information technology and the modalisation of organizational behaviour: A theoretical framework. Journal of Information Technology, 21, 86-98. [E-Reserve]

Devaraj, S., & Kohli, R. (2003). Performance impacts of information technology: Is actual usage the missing link? Management Science, 49(3), 273-289. [Read Sections 1, 2, and 3.4 and skim the rest.] [Online]

Jacoby, G. A. (2007). Intranet model and metrics. Communications of the ACM, 50(2), 43-50. [E-Reserve]

Kohli, R., & Sherer, S. A. (2002). Measuring payoff of information technology investments: Research issues and guidelines. Communications of the AIS, 9, 241-268. [Online]

Tynan, D. (2005, November). Proving your project's worth. InfoWorld, 27(47), 24-29. [E-Reserve]

Optional Reading:

Hong, W., Thong, J. Y.L., & Tam, K. Y. (2007). How do web users respond to non-banner-ads animation? The effects of task type and user experience, Journal of the American Society for Information Science & Technology, 58(10), 1467-1482. [E-Reserve]

McCoy, S., Everard, A., Polak, P., & Galletta, D. F. (2007). The effects of online advertising. Communications of the ACM, 50(3), 84-88. [E-Reserve]

Tillquist, J. (2000). Institutional bridging: How conceptions of IT-enabled change shape the planning process. Journal of Management Information Systems, 17(2), 115-154. [E-Reserve]



Week 11: Understanding survey research /service quality

Babbie, E. (2004). The practice of social research (10th ed.). Belmont, CA: Wadsworth/Thomson Learning. Chapter 9.

Nielsen, J. (March 1, 2004). Risks of quantitative studies. Retrieved December 30, 2006. [Online]

Ryan, J., McClure, C. R., & Bertot, J. C. (2001). Choosing measures to evaluate networked information resources and services: Selected Issues. In C. R. McClure & J. C. Bertot (Eds.), Evaluating networked information services: Techniques, policy and issues (pp. 111-135). Medford, NJ: Information Today Inc. [E-Reserve]

UCLA Library Instructional Services Advisory Committee. (2001, Spring). UCLA Library Information Competence at UCLA: Report of a Survey Project. (Read esp. Section III, Appendix A, Appendix B [instrument], and Appendix C.) [Online]

Choose one between the two:

Hernon, P., & Calvert, P. (2005). E-serve quality in libraries: Exploring its features and dimensions. Library & Information Science Research, 27(3), 377-404. [E-Reserve]

Negash, S., Ryan, T., & Igbaria, M. (2003). Quality and effectiveness in web-based customer support systems. Information & Management, 40(8), 757-768. [E-Reserve]

Optional Reading:

Association for Research Libraries. (2005). ARL new measures initiatives. Retrieved December 7, 2007. [Online]

Gunn, H. (2002). Web-based surveys. Changing the survey process. Retrieved September 8, 2007. [Online]

Parasuraman, A., Zeithaml, V. A., & Malhotra, A. (2005). E-S-QUAL: A multiple-item scale for assessing electronic service quality. Journal of Service Research, 7(3), 213-233. [Online]

Week 12: Understanding qualitative research

Kaplan, N., Chisik, Y., Knudtzon, K., Kulkarni, R. et al. (2004, June). Supporting sociable literacy in the international children's digital library. Proceeding of the 2004 conference on interaction design and children: Building a community. [Online]

Kelly, D., et al. (2007). Using interview data to identify evaluation criteria for interactive, analytical question-answering systems. JASIST, 58(7), 1032-1043. [Online]

Salvador, T., & Anderson, K. (2003). Practical Considerations of Context for Context Based Systems: An Example from an Ethnographic Case Study of a Man Diagnosed with Early Onset Alzheimer's Disease.[E-Reserve]

Optional Reading:

Haynes, S. R., Purao, S., Skattebo, A. L. (2004). Situating evaluation in scenarios of use. Proceedings of the 2004 ACM conference on Computer Supported Cooperative Work, 92 - 101. [E-Reserve]

Mantei, M. M., & Teorey, T. J. (1989). Incorporating behavioral techniques into the systems development life cycle. MIS Quarterly, 13(3), 257-274. [E-Reserve]

Westbrook, L. (1994). Qualitative research methods: A review of major stages, data analysis techniques, and quality controls. Library & Information Science Research, 16(3), 241-254. [E-Reserve]

Vasconcelos, A. C. (2007). The use of grounded theory and of arenas/social worlds theory in discourse studies: A case study on the discursive adaptation of information systems. The Electronic Journal of Business Research Methods, 5(2), 125-136. [Online]


Week 13: Introduction to data analysis

Salkind, N. J. (2007). Chapters 2, 3, 4, & 5.

Week 14: More on data analysis

Davidson Chapter 7

Salkind, N. J. (2007). Chapters 9, 10, & 11.

Week 15: Practical considerations in evaluation & Politics of evaluation

Davidson Chapter 10

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2003). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). New York: Allyn & Bacon. [E-Reserve]

Wilson, M., & Howcroft, D. (2000). Power, politics, and persuasion: A social shaping perspective on IS evaluation. In 23rd IRIS (Information systems research seminar in Scandinavia) Conference, L. Svensson, U. Snis, C. Sorensen, H. Fagerling, T. Lindroth, M. Magnusson, and C. Ostlund, C. (Eds.), pp. 725-739. [E-Reserve]

Optional Reading:

Ravichandran, T., & Rai, A. (2000). Quality management in system development: An organizational system perspective. MIS Quarterly, 24(3), 381-415. [E-Reserve]

Home | About | Course Readings | Syllabus | Handouts | Assignment

URL: https://ella.slis.indiana.edu/~nhara/teaching/sp08/s519/read.html

Contact Noriko at nhara [at] indiana.edu