Our 2 year goal, by the end of AY 24-25, is to produce GAI-enabled course structure across the introductory STEM curriculum.
The tools should be agnostic to the textbook(s) used in the course, and should be implementable with a minimal time investment in courses nationwide.
Initial backend is likely to be ChatGPT, currently at version 4, but we should strive to build an architecture that can evolve and adapt to new and better and different tools. We need to have an overall architecture philosophy evolve with our development spirals.
This effort will take place in stages, as follows:
Summer 2023 - establish structure for the process, secure resources, build team, delineate clear goals for the Fall 2023 term, define a subset of experiments for Fall 2023. Initial tool development for API-interface exploitation.
Develop an experimental active learning lecture series that incorporates GPT-4 capabilities and measure its effectiveness in student comprehension and retention.
Devise 8 experiments to do in Fall 2023
First development sprint for 8 experiments
Fall 2023 - Make all STEM courses and instructors at least GAI-aware.
Conduct initial set of 8 GAIPedagogy experiments across the STEM curriculum. Tracking of pedagogical experimental outcomes.
Oct 2023 workshop for Harvard STEM faculty.
Planning and invitations for GAISTEMP-1 conference.
Development sprint for API-interface tools
January 2024- Three day national Workshop on STEM GAI pedagogy.
Development sprint for GAI active learning and HW tools.
First draft of GAISTEMP experimental-results paper from first 8 examples
Spring 2023 - General Education course offered.
Rollout of prototype active learning and HW modules in 4 intro STEM courses, with assessments.
Summer 2024 - Extension of prototype active learning and HW modules to all of intro STEM curriculum, with assessments.
Development sprint for course modules.
Fall 2024 - First offering of GAI-empowered courses across entire introductory Harvard STEM curriculum. Prepare for second national workshop.
January 2025 - Second national workshop: GAISTEMP-2.
Spring 2025 -
January 2025 - Third national workshop: GAISTEMP-3.
Summer 2025
Management structure and resources.
We will coordinate and perhaps embark on joint projects with MIT. We will designate an MIT liaison subgroup. Cadence of those meetings TBD.
We will coordinate across the various elements of the Harvard community by participating in quarterly stakeholder meetings.
Our divisional coordinator will be Assistant Dean for Science Education Logan McCarty.
We will engage undergraduates, graduate students, postdocs, staff, and faculty in this effort.
Our primary perspective will be to support faculty in the incorporation of GAI tools into our learning program.
Fall term 2023
Instructional methodology viewpoint: (pick 8 experiments for Fall 2023)
| Item | experiments | courses | lead | GPT aspect needed | Validation criteria |
|---|---|---|---|---|---|
A: Incorporation of GAI into lecture-format STEM learning:
|
| ||||
| B: Develop and Exploit short-cycle adaptive problem sets with real-time feedback | |||||
| C: Assist with analysis and gain insights from lab data. | |||||
| D: interactive student self-assessments. | |||||
| E: In-class group consultation (peer instruction) with ChatGPT participation | |||||
| F: capturing and submitting GAI-enabled work for evaluation by course staff | |||||
| G: automated evaluation of understanding of material, by evaluating answers to questions we provide. | |||||
| H: Ascertain subject-level mastery needed to exploit natural-langauge-driven code development. Try out a non-analytic problem and assess the results. |
| 15 a,b,c | numerical solution code | ||
| I: incorporate into HW and assessments the ability to perform calculations, as pioneered by Khan Academy | arithmetic capability | ||||
| J: incorporate course-specific training inputs and give that high weighting | custom training inputs | ||||
| K: Automation grading and assessments of student competence. | sequential prompts run open loop, no adjustment | ||||
| L: dynamic tutoring | sequential prompts with iterative adjustment | ||||
| M: GAI assisted generation and refinement of course instructional and assessment materials- HW, exams, quizzes, etc. |
|
| respondent | notes | |
|---|---|---|
| Carlos Argueles | carguelles@fas.harvard.edu | Hola Chris: Thanks for sending this email. I am very excited about this new technology. In fact, with Louis Deslauriers, we used some of it in my Physics 15B last semester. Carlos |
Course-based viewpoint
- Assess our assessments: run midterm and final exams of science courses through GPT-4 and grade the results. Compare to overall student performance.
- Enhance our assessments. Solicit constructive feedback on the exam questions we submit.
- Assess our homework: run homework assignments through GPT-4 and grade the results. Compare to overall student performance.
- Enhance our assignments. Solicit constructive feedback on the homework we submit.
- Ask (require?) students to use GPT-4 on selected assignments to get feedback and examples of how it can be used.
- Course-specific chat-bots- what training data?
- For large lecture classes- merge active learning with GPT
- For sections- aggregation of questions,
- For labs- try out data analysis methods and inference
- Customized training assembly of material - what do we need to start to capture?
- Khan academy like adaptive tutorials
- How does this shift the workload in our non-ladder teaching capacity? Especially sections and TFs and grading?
Curriculum-based viewpoint
- What are the high-level learning goals for our students, in a GAI world?
- How can we critical thinking skills and a truth-validation mindset?
- What progression of instruction and course expectations are appropriate?
Student-centric viewpoint
- Learning how to craft a prompt that gets what you want
- GAI as a consultant
- GAI for self-assessment
- iterative refinements in GAI interactions
- Learning how to validate and verify results
- Honing critical thinking skills in the GAI context.
- Ethical, responsible, thoughtful use of powerful tools.
- Accommodating disabilities and ensuring equitable access.
IT-centric viewpoint
- How do we integrate these learning tools with existing platforms? Examples include Canvas, grade sheets, Sharepoint, Jupiter notebooks, Open OnDemand, data repositories, assignments, work-uploading tools, etc?
- What is the best approach to licensing and token-purchasing?
- How do we throttle and regulate non-course abuse?
- How do we develop, curate, and support the use of this new toolkit?
- What are institutional roles, responsibilities, accountabilities, and authorities?
- What staffing is needed, at what levels of the organization?
- Who pays for what?
Specific examples :
Modulated Friction example
Spring term 2024
Gen Ed 1188 https://gened.fas.harvard.edu/classes/catching-tsunami-riding-gpt-wave
GAISTEM core team
| Who? | From? | |
|---|---|---|
| Christopher Stubbs | FAS Sci Div | stubbs@g.harvard.edu |
| Logan McCarty | FAS Sci Div | mccarty@fas.harvard.edu |
| Greg Kestin | FAS Sci Div | kestin@fas.harvard.edu |
| Erin Collins | FAS Sci Div | erin_collins@fas.harvard.edu |
| Jefferson Burson | HUIT | jefferson_burson@harvard.edu |
| Ventz Petkov | HUIT | ventz_petkov@harvard.edu |
| David LaPorte | HUIT | david_laporte@harvard.edu |
| Colin Murtaugh | HUIT | colin_murtaugh@harvard.edu. |
Departmental
GAISTEM stakeholder group, quarterly meetings
| stakeholder | representative |
|---|---|
| College OUE | Amanda Claybaugh Anne Harrington |
| Harvard College | Rakesh Khurana |
| Bok Center | Tamara Brenner, Adam Beaver |
| HGSE | |
| humanities div. | Robin Kelsey Jeffrey Schnapp |
| social sci div. | |
| VPAL | Bharat Anand |
| HUIT | Klara Jelinkova |
Links
https://bokcenter.harvard.edu/artificial-intelligence Bok center AI page
https://science.fas.harvard.edu/chatgpt divisional resource page