OHSU Online · School of Medicine
Design & Eval in Health Inform
- Winter 2019
- 3 Credits
- 01/07/2019 to 03/22/2019
- Modified 01/03/2019
This course examines a variety of design and evaluation paradigms and methods in both research as well as clinical informatics domains, and will introduce you to basic concepts associated with informatics project design and evaluation. The breadth of material covered is large, and hence the curriculum for this course is designed at a higher level than a course that examines a single domain of informatics methods in detail.
Yes, there is a prerequisite requirement for this course. You will need to successfully complete a prior course in biostatistics such as BSTA 525 (formerly PHPM 524), or BSTA 511, or MATH 530, or equivalent, before enrolling for BMI 560/660.
Homepage: My OHSU web page is at: http://goo.gl/3C1D
Address: BICC 409
Department of Medical Informatics and Clinical Epidemiology
Oregon Health & Science University
3181 Sam Jackson Park Rd
Portland, OR 97239
Email: [email protected]
Telephone: 503 494 4469, Google Voice: 503 208 4568
How to reach me: Email is always the best way to communicate with me.
My office hours are by appointment – email for a time to meet virtually or face-to-face.
This is a fully online course. Materials are released every Monday at 8 a.m., PST.
Students learn through the Sakai learning management software at https://sakai.ohsu.edu. The online component includes lectures, PowerPoint slides and handouts, reading assignments, and project material. If you have any technical questions or if you need help logging in, please contact the Sakai Help Desk, which is open Mon – Fri, 8 am – 9 pm, weekends from 12 pm – 5 pm (closed on official OHSU holidays).
Sakai Help Desk Contact Information:
(Email) [email protected]
Title: Practical Research: Planning and Design (12th Edition)
Authors: Paul D Leedy and Jeanne Ellis Ormrod
Paperback: 408 pages
Publisher: Pearson; 12th edition (February 19, 2018)
Course Competencies, Outcomes, and Objectives
This course will:
- Provide students with an overview and framework for the design and evaluation of biomedical informatics projects, across a spectrum of system types, developmental stages, user perspectives, and evaluation methods.
- Examine in detail a variety of design and evaluation methods, and apply them to specific projects in four key categories: quantitative evaluation, qualitative evaluation, software development, and business-oriented planning.
- Prepare students to design and conduct thesis or capstone research/development projects in biomedical informatics.
The grading framework for this course is:
Assignments and other deliverables, including a end-of-term paper = 40%
Online participation = 40%
Tests and Exams = 20%
Grades are assigned on a curve, but are generally based on the following cutoffs:
This course features an end-of term final examination, which is taken online. Please note that as per OHSU policy, you will be required to take this exam under the supervision of an online proctoring service.
The exam is a mandatory element of this course, and students are encouraged to contact the Sakai Help Desk before signing up for this course if they have any questions regarding the policy or the online proctoring method that is utilized.
How will online participation be graded?
Each week, students will be required to participate in discussions, held in the Forums section on Sakai. Discussions will typically center on a topic that I will put up at the beginning of each week. We will also discuss group projects, individual student final projects, and other CIS-related topics that you may want to talk about during the term.
Forums participation is graded using the following rubric:
0 = no participation
1 = minimal participation
2 = adequate participation in discussion (students posts satisfy the requirements)
3 = student post demonstrates that they have met learning objectives
4 and 5 = superior participation that exceeds the required response (participation in a meaningful manner that promotes the learning of others)
You will see that there is a significant weightage towards participation that allows others to learn from your perspectives.
What are students expected to do in order to successfully complete this course?
To succeed in this course, students are expected to:
- Review reading assignments; including lectures, chapters from the required textbook, articles or handouts, and engage in independent reading for projects and questions. Students are responsible for learning all content in the assigned readings, whether discussed in the lectures or not.
- Participate in discussions; this class actively encourages collaboration, engagement and participation by all students. The student's participation grade is based on their level of contribution to the learning of others.
- Complete assignments and other deliverables in a timely fashion. It is anticipated that students will substantiate their assertions with appropriate attribution, and demonstrate that they possess the ability to critically retrieve and analyze information that is available in the informatics literature. Some assignments and projects may involve teamwork for preparation, presentation, and grading. As is common in real life, there are penalties for late submissions and bonuses for those who turn their work in early.
- Complete quizzes and other evaluative tools during the course in a timely fashion.
This course offers an intensive amount of information and it is highly recommended that students develop their own routine schedule to keep up with course materials; it is likely that students will not be able to catch up if they miss a substantial number of lectures and assignments over the duration of the course.
Please note that this course features a final exam that will be an online closed-book, timed effort, with questions that will be in a multiple-choice format.
What is your philosophy on learning and the role of the instructor?
At this level of education, I anticipate that students will be adult learners and already have significant insight into the way they learn best.
My classes emphasize a culture where participation and sharing information results in the dissemination of knowledge and wisdom. Expect to spend a significant proportion of your time engaged in online conversation with other students and with me.
When I teach a course at the graduate level, I often find that I learn as much from you as you do from your instructors. I see my role as primarily that of a facilitator, helping you reach your learning goals for the course.
What is the best way to participate in Forums?
Participation scores are graded weekly, so read and post on the Forum regularly! The weekly score is based on your level of contribution to the learning of others. I’m looking for quality, not quantity! It’s not how much you post, but what you say and how you say it.
Remember that a Forum format is just that – a place for meaningful discussion. Responses to posts often are as useful to enhance learning as the original post itself. The Forum is a great format to help us learn from each other.
The course is structured in a modular fashion as a series of weekly components. Each weekly module includes a lecture, additional reading material, and assignments. Tests may also be held online. Lectures may be delivered by guest lecturers with experience in research or in the operational informatics universe. Guest lecturers may also participate in in-class sessions (for in-class instances of the course).
Example of the course schedule and topics covered (note that the actual schedule for this course may vary from the example):
Overview of the course
Software methods part 1: Design, Usability
Software methods part 2: Evaluation
Quantitative methods part 1: Design
Quantitative methods part 2: Evaluation
Mixed methods, Additional topics
Student projects, Finals week
School Policies and Resources
Graduate Studies Guidelines:
Students are responsible for following all OHSU School of Medicine, Graduate Studies, and program/department guidelines & policies. For more information, please visit here. For program/department guidelines & policies, please inquiry with the program/department director and/or coordinator.
School of Medicine Conduct Policy (housed under the graduate studies guidelines section)
Students are responsible for their own academic work. Students are expected to have read and practice principles of academic honesty, as presented in the Graduate Studies Student Handbook.
The School of Medicine reserves the privilege of retaining only those students who, in the judgement of the faculty, satisfy the requirements of scholarship and clinical performance necessary to maintain the highest standards. The Student Handbook has information about academic standards and probation and dismissal policies.
Grading Criteria, Academic Standards, & Release of Final Grades:
Graduate Studies in the OHSU School of Medicine is committed to providing grades to students in a timely manner. Course instructors will provide students with information in writing at the beginning of each course that describes the grading policies and procedures including but not limited to evaluation criteria, expected time needed to grade individual student examinations and type of feedback they will provide.
All coursework applied towards degree requirements must meet the minimum cumulative grade point average of at least 3.0.
Refer to the School of Medicine Graduate Studies Forms & Policies for withdraw, incomplete, and in-progress grading standards. Final course grades will be posted with the OHSU Registrar the Monday following the last day of the term. On those occasions when a grade has not been submitted by the deadline, the following procedure shall be followed:
- The Department*/Program** Coordinator will immediately contact the Instructor requesting the missing grade, with a copy to the Program Director and Registrar.
- If the grade is still overdue by the end of next week, the Department*/Program** Coordinator will email the Department Chair directly, with a copy to the Instructor and Program Director requesting resolution of the missing grade.
- If, after an additional week the grade is still outstanding, the student or Department*/Program** Coordinator may petition the Office of Graduate Studies for final resolution.
*For courses that are run by a specific department.
**For the conjoined courses (course number is preceded by CON) that are run by Graduate Studies.
Graduate Studies Inclement Weather Procedures
Inclement weather procedures can be found here. In the case of inclement weather, the faculty member will email or place a voice-mail greeting on her/his office telephone number by 6:00am on the day of the clinical or class to give instructions to students about the class schedule.
Graduate Studies Copyright Information
Every reasonable effort has been made to protect the copyright requirements of materials used in this course. Class participants are warned not to copy, audio, or videotape in violation of copyright laws. Journal articles will be kept on reserve at the library or online for student access. Copyright law does allow for making one personal copy of each article from the original article. This limit also applies to electronic sources.
DMICE Communication Policy
- If the syllabus directs the student to contact the TA before contacting the instructor, the student should do so. Otherwise, the student should contact the instructor and allow 2 business days (not including weekends) for a response.
- If the student does not receive a response from the instructor within 2 business days, s/he should contact the TA (if there is one). When contacting the TA s/he should cc the instructor and Diane Doctor at [email protected].
- If a student does not receive a response from the TA within 1 business day (not including weekends), s/he should contact Diane Doctor at [email protected] and cc the instructor and the TA.
- If Diane does not reply within 1 business day (not including weekends), the student should contact Andrea Ilg at [email protected].
- Students having difficulties with Sakai should contact the Sakai Help Desk at [email protected] or at (877) 972-5249. Sakai help is available M-F from 8am to 10-pm and weekends from Noon to 5pm. Do not contact the instructor.
When Problems Arise
It is critical to contact the appropriate person when problems arise:
- For basic Sakai problems and course issues (e.g., cannot log in, after-hours technical assistance, Course Materials or Forum not available/accessible during regular business hours/days), contact the Sakai Help Desk: Toll-Free - (877) 972-5249; email - [email protected] Sakai help is available M-F from 8 am to 9 pm and weekends from Noon to 5pm.
- For questions about course content (e.g., do not understand a topic or disagree with homework quiz answer), contact the Teaching Assistant, who will be announced at the beginning of the course: go to the Email Tab after logging into the course and choose “Associate” role to send message to the TA or post a question in the Forums.
It is OHSU policy that any exam offered online and worth more than 10% of the final course grade must be virtually proctored. In this course, we will be using the services of Examity, a remote proctoring services company. You will be required to schedule your exam three (3) weeks ahead of time. There is no cost to the student. More information will be provided to you regarding setup, scheduling, and requirements in the Course Materials.
Turn It In
In an effort to uphold the principles and practice of academic honesty, faculty members at OHSU may use originality checking systems such as Turnitin to compare a student’s submitted work against multiple sources. To protect student privacy in this process, it will be necessary to remove all personal information, i.e. student name, email address, student u-number, or any other personal information, from documents BEFORE submission.
Sakai and TLC Help Desk
You will learn through the Sakai learning management software at http://sakai.ohsu.edu. The online component includes reading material, lectures (including streaming presentations and handouts), project material, learning assignments, and online discussions. If you have any technical questions or if you need help logging in, please contact the Sakai Help Desk, which is open Mon – Fri, 8 am – 9 pm and weekends 12 pm – 5 pm, Pacific Time.
(Email) [email protected]
Please use professional etiquette when communicating with peers and the instructor. This means avoiding aggressive or offensive language, showing respect for others’ opinions and positions, and conducting yourself as if you were face to face with them. Please pay special attention to etiquette in class forums and when using email. If you notice someone violating this policy, please make the instructor and TA aware of the problem.
OHSU Graduation Core Competencies
1. Professional Knowledge and Skills
Demonstrate competence in the core knowledge, skills, and practices as defined by degree programs and relevant professional licensing and credentialing boards.
2. Reasoning and Judgement
Demonstrate the ability to identify and define problems, critically compare options, make timely decisions or recommendations, identify uncertainties, and use findings to improve outcomes in light of evolving evidence.
3. Evidence-Based Practice and Research
Demonstrate the ability to access, evaluate, and apply relevant science knowledge to support evidence-based health care, disease prevention, health promotion and discovery.
4. Lifelong Learning
Demonstrate the ability to recognize gaps in knowledge and experience through informed self-assessment and reflective practices, and take actions to address those gaps.
Demonstrate active listening and oral and written communication skills with diverse individuals, communities, and colleagues to ensure effective, culturally appropriate exchange of information.
6. Professionalism and Ethics
Demonstrate integrity, honesty, knowledge of ethical principles and the standards of professional conduct, and the ability to apply ethical principles in clinical care, research, education or community service.
Demonstrate the abilities required to foster and work effectively within collaborative, team-based environments.
8. Safety and Quality Improvement
Demonstrate the ability to identify situations that compromise safety and participate in risk reduction and continuous quality improvement.
Demonstrate an appropriate understanding of evolving health care systems, health and science policy, and resource allocation in order to optimize human health and scientific discovery.
10. Patient/Client-Centered Care
Additionally, clinical degree program graduates will be able to...
Demonstrate the ability to collaborate with diverse individuals, families, and communities to provide quality care that is respectful of and responsive to their preferences, needs, attitudes, beliefs and values.
Institutional Policies and Resources
Statement Regarding Students with Disabilities:
OHSU is committed to inclusive and accessible learning environments in compliance with federal and state law. If you have a disability or think you may have a disability (mental health, attention-related, learning, vision, hearing, physical or health impacts) contact the Office for Student Access at (503) 494-0082 or OHSU Student Access to have a confidential conversation about academic accommodations. Information is also available at Student Access Website. Because accommodations may take time to implement and cannot be applied retroactively, it is important to have this discussion as soon as possible.
Student Evaluation of Courses:
Course evaluation results are extremely important and used to help improve courses and the learning experience of future students. Responses will always remain anonymous and will only be available to instructors after grades have been posted. The results of scaled questions and comments go to both the instructor and their unit head/supervisor. Refer to Student Evaluation of Courses and Instructional Effectiveness, *Policy No. 02-50-035.
*To access the OHSU Student Evaluation of Courses and Instructional Effectiveness Policy, you must log into the OHSU O2 website.
Copyright laws and fair use policies protect the rights of those who have produced the material. The copy in this course has been provided for private study, scholarship, or research. Other uses may require permission from the copyright holder. The user of this work is responsible for adhering to copyright law of the U.S. (Title 17, U.S. Code). To help you familiarize yourself with copyright and fair use policies, the University encourages you to visit its Copyright Web Page
Sakai course web sites contain material protected by copyrights held by the instructor, other individuals or institutions. Such material is used for educational purposes in accord with copyright law and/or with permission given by the owners of the original material. You may download one copy of the materials on any single computer for non-commercial, personal, or educational purposes only, provided that you (1) do not modify it, (2) use it only for the duration of this course, and (3) include both this notice and any copyright notice originally included with the material. Beyond this use, no material from the course web site may be copied, reproduced, re-published, uploaded, posted, transmitted, or distributed in any way without the permission of the original copyright holder. The instructor assumes no responsibility for individuals who improperly use copyrighted material placed on the web site.
Syllabi Changes and Retention:
Syllabi are considered to be a learning agreement between students and the faculty of record. Information contained in syllabi, other than the minimum requirements, may be subject to change as deemed appropriate by the faculty of record in concurrence with the academic program and the Office of the Provost. Refer to the *Course Syllabi Policy, 02-50-050.
*To access the OHSU Course Syllabus Policy, you must log into the OHSU O2 website.
Commitment to Diversity & Inclusion:
OHSU is committed to creating and fostering a learning and working environment based on open communication and mutual respect. If you encounter sexual harassment, sexual misconduct, sexual assault, or discrimination based on race, color, religion, age, national origin, veteran’s status, ancestry, sex, marital status, pregnancy or parenting status, sexual orientation, gender identity, disability or any other protected status please contact the Affirmative Action and Equal Opportunity Department at 503-494-5148 or [email protected]. Inquiries about Title IX compliance or sex/gender discrimination and harassment may be directed to the OHSU Title IX Coordinator at 503-494-0258 or [email protected]
Modified Operations, Policy 01-40-010:
Students should review the Student Portal, O2, or call OHSU’s weather alert line at 503-494-9021 for the most up-to-date information on OHSU-wide modified operations which include but are not limited to delays or closures for inclement weather. For specific campus information, outside of Portland, check the following sites:
OHSU Resources Available to Students*:
Mackenzie Hall, Rm. 1120
503-494-7800; Email the Registrar
Student Registration Information:
To Register for Classes
OHSU ITG Help Desk
Regular staff hours are 6 a.m. to 6 p.m., Monday through Friday, but phones are answered seven days a week, 24 hours a day. Call 503 494-2222.
Teaching and Learning Center
Academic Support Counseling and Sakai Course Management System, please contact the TLC Help Desk at 877-972-5249 or email TLC Help Desk
Concourse Syllabus Management
For help with accessing your Concourse Syllabus: Please contact the Sakai help Desk for all other Concourse inquiries please visit the Concourse Tutorial Website or please contact the Mark Rivera at [email protected] or call 503-494-0934
- Emergency on Campus: 503-494-4444 (Portland)
- Non-emergency: 503-494-7744; Contact Public Safety
- For Emergencies dial 911
- Officer Assistance: (541) 552-6911
- Emergency: 503-838-9000
- Main Office (Open 24/7): 503-838-8481
- Emergency: 911
- Camus Security Non Emergency: 541-962-3911
- Emergency: 911 or 541-885-0911
- Camus Security Non Emergency: 541-885-1111
*Joseph B. Trainer Health & Wellness Center
Baird Hall, Rm. 18 (Primary Care) and Rm. 6 (Behavioral Health)
503-494-8665; For urgent care after hours, 503-494-8311 and ask for the Nurse on call.
Wellness Center Information
Wellness Center Website
*Exceptions include Public Health students who have selected PSU as their primary providers and students whose home campus is a satellite campus or online. If your home institution is not on the Portland campus, contact your home institution student support services for more information.
Library: Biomedical Information Communication Center
BICC Library Hours of Operation
Is this a required course?
If you are a student in the masters or PhD program, you are required to take BMI 560 (or BMI 660, the version of this course for PhD students).
How does this course fit into the DMICE curriculum?
The DMICE curriculum includes an Evaluative Science branch that is intended to enable informatics students understand the fundamental aspects of scientific research including statistics, qualitative research methods, epidemiology and health data analysis.
BMI 560/660 is one of two core courses in the Evaluative Sciences branch of the curriculum - the other is an introductory biostatistics course.
This branch also includes a third elective course in evaluation methods that is chosen by the student according to their individual interests and needs (examples include Qualitative Research Methods, Quantitative Research Methods, Software Engineering, or other evaluation courses approved by the student’s advisor). Please note that some of these in-depth methods courses offered by DMICE may require BMI 560/660 as a prerequisite.
Also, please note that if you have already successfully taken several methods-intensive courses, you may not be required to take BMI 560/660.
The DMICE curriculum also delineates a pathway that will prepare students to successfully complete a capstone or thesis project. This course is the second in a three-course series, which also includes BMI 515 (Ethical, Legal and Social Issues in Biomedical Informatics) and BMI 570 (Scientific Writing and Communication for Informatics Students) that prepare first-year masters students to conduct a capstone or thesis project in year two. This course is also an excellent introductory course if you are planning the direction of your research for your PhD.
What makes informatics projects unique?
Informatics projects in the research environment, as well as those in the operational arena, often display a wide spectrum of disparity. Examples of informatics projects are quite varied; from field studies that improve the understanding of tasks and information needs of users, to development projects that design, build, and deploy clinical information systems, to studies that assess the impact of information systems on health care processes and outcomes.
Why is the process of designing an informatics project challenging?
Informaticians are often asked to help design and implement projects in diverse settings that require frameworks to be drawn from different disciplines. Additionally, for their capstone or thesis, informatics students often design projects that traverse several traditional branches of learning.
How will this course help informaticians?
At the most fundamental level, this course provides you with a "toolkit" of design and evaluation concepts that will allow you to build your own projects, both in research as well as the clinical informatics realms.
This course is intended to provide a high-level overview of the concepts, terminology, and strategies needed to design and evaluate projects in biomedical informatics, including methodologies drawn from software engineering, qualitative research, quantitative research, and business administration.