Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Our 2 year goal, by the end of AY 24-25, is to produce GAI-enabled course structure across the introductory STEM curriculum.
The tools should be agnostic to the textbook(s) used in the course, and should be implementable with a minimal time investment in courses nationwide.
Initial backend is likely to be ChatGPT, currently at version 4, but we should should  strive to build an architecture that can evolve and adapt to new and better and different tools. We need to have an overall architecture philosophy evolve with our development spirals.  

This effort will take place in stages, as follows:

Summer 2023 - establish structure for the process, secure resources, build team, delineate clear goals for the Fall 2023 term, define a subset of experiments for Fall 2023. Initial tool development for API-interface exploitation.
                           Develop an experimental active learning lecture series that incorporates GPT-4 capabilities and measure its effectiveness in student comprehension and retention. 
                           Devise 8 experiments to do in Fall 2023
                           First development sprint for 8 experiments

...

January 2025 - Second national workshop on : GAISTEMP-2. 

Spring 2025 - 

January 2025 - Third national workshop: GAISTEMP-3. 

Summer 2025

Management structure and resources. 

We will coordinate and perhaps embark on joint projects with MIT. We will designate an MIT liaison subgroup. Cadence of those meetings TBD. 
We will coordinate across the various elements of the Harvard community by participating in quarterly stakeholder meetings. 
Our divisional coordinator will be Assistant Dean for Science Education Logan McCarty. 
We will engage undergraduates, graduate students, postdocs, staff, and faculty in this effort. 
Our primary perspective will be to support faculty in the incorporation of GAI tools into our learning program. 

Fall term 2023

Instructional methodology viewpoint:

...

  (pick 8 experiments for Fall 2023)

ItemexperimentscoursesleadGPT aspect neededValidation criteria

A: Incorporation of GAI into lecture-format STEM learning: 

  1. Develop experimental active learning lecture modules that incorporate GAI capabilities, and devise methods to measure their effectiveness in student comprehension and retention. 
  2. Develop first-generation tools 
  1. synthesis of student-provided questions, in real time
  2. open-response quizzes rather than multiple choice
  3. peer-instruction including GAI
  4. Sequential interaction on course prompts






B: Develop and Exploit short-cycle adaptive problem sets with real-time feedback




C: Assist with analysis and gain insights from lab data. 




D: interactive student self-assessments. 




E: In-class group consultation (peer instruction) with ChatGPT participation




F: capturing and submitting GAI-enabled work for evaluation by course staff




G: automated evaluation of understanding of material, by evaluating answers to questions we provide. 




try H: Ascertain subject-level mastery needed to exploit natural-langauge-driven code development.
         Try out a non-analytic problem and assess the results. 
  1. modulated-friction example. Modulated Friction example

15 a,b,c
Math 22


numerical solution include code
I: incorporate into HW and assessments the ability to perform calculations, as pioneered by Khan Academy


arithmetic capability
J: incorporate course-specific training inputs and give that high weighting


custom training inputs
K: Automation of grading and assessments of student competence. 


sequential prompts run open loop, no adjustment
L: dynamic tutoring


sequential prompts with iterative adjustment 
Generation M: GAI assisted generation and refinement of course instructional and assessment materials- HW, exams, quizzes, etc. 
  1. request a critique of exam questions
  2. request answer key to HW and exam questions




...

  1. Assess our assessments: run midterm and final exams of science courses through GPT-4 and grade the results. Compare to overall student performance. 
  2. Enhance our assessments. Solicit constructive feedback on the exam questions we submit. 
  3. Assess our homework: run homework assignments through GPT-4 and grade the results. Compare to overall student performance. 
  4. Enhance our assignments. Solicit constructive feedback on the homework we submit. 
  5. Ask (require?) students to use GPT-4 on selected assignments to get feedback and examples of how it can be used. 
  6. Course-specific chat-bots- what training data? 
  7. For large lecture classes- merge active learning with GPT
  8. For sections- aggregation of questions, 
  9. For labs- try out data analysis methods and inference
  10. Customized training assembly of material - what do we need to start to capture?
  11. Khan academy like adaptive tutorials
  12. How does this shift the workload in our non-ladder teaching capacity? Especially sections and TFs and grading? 

Curriculum-based viewpoint

  1. What are the high-level learning goals for our students, in a GAI world?
  2. How can we critical thinking skills and a truth-validation mindset?
  3. What progression of instruction and course expectations are appropriate?   

Student-centric viewpoint

...

C. Stubbs
L. McCarty
G. Kestin

Departmental 

GAISTEM stakeholder group, monthly quarterly meetings

stakeholderphysicsMatt Schwartz
Louis Deslauriers
statisticsXiao-Li Meng
Lucas Janson
EPSBrandon MeadeMCBSean EddyOEBMichael DesaiHEB?SCRB?CCB?AstronomyDoug FinkbeinerSEAS

Martin Wattenberg
Scot Martin
Jim Waldo
Rebecca Neeson

MathCliff Taubes? representative
College OUEAmanda Claybaugh
Anne Harrington
Harvard CollegeRakesh Khurana
Bok CenterAdam Beaver
HGSE
humanities div. Robin Kelsey
Jeffrey SchnappsSchnapp
social sci div. 
VPALBharat Anand
HUITKlara Jelinkova

...