Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Our 2 year goal, by the end of AY 24-25, is to produce GAI-enabled course structure across the introductory STEM curriculum.
The tools should be agnostic to the textbook(s) used in the course, and should be implementable with a minimal time investment in courses nationwide.
Initial backend is likely to be ChatGPT, currently at version 4, but we should  strive to build an architecture that can evolve and adapt to new and better and different tools. We need to have an overall architecture philosophy evolve with our development spirals.  

...

ItemexperimentscoursesleadGPT aspect neededValidation criteria

A: Incorporation of GAI into lecture-format STEM learning: 

  1. Develop experimental active learning lecture modules that incorporate GAI capabilities, and devise methods to measure their effectiveness in student comprehension and retention. 
  2. Use multi-shot learning to have GAI produce 'clicker' questions and active learning modules.
  3. Have GAI do assessments of online active learning modules. 
  4. Increase accessibility of course materials
  1. Assess utility of synthesis of student-provided question content, in-class just-in-time learning
  2. Assess utility of synthesize summaries of open-response clicker questions rather than multiple choice, for adaptive lectures
  3. Assess utility of peer-instruction that includes GAI virtual partners
  4. Assess utility of GAI-produced active learning modules
  5. Compare restricted-content (textbook and lecture materials) vs. all-content GAI generated active learning modules. 




shared-edit document fed into GAI, results available to instructor. 

Multi-shot learning using restricted content and open 

PPT and math content upload as tokens


B: Develop and Exploit short-cycle adaptive problem sets with real-time feedback

  1. produce a 'virtual tutor' that uses a script of desired learning goals to promote student learning
  2. distinguish between conceptual mastery and ability to solve quantitative problems. To what extent can GAI do the math part? 
  1. Assess learning effectiveness of rapid-feedback interactive learning, outside of lecture. 
  2. assess zero-shot vs multishot training for virtual tutor effectiveness. 
  3. Assess instructor burden compared to current methods. 


Ability to identify main topics, and embed in GAI interaction. 

Arithmetic needs to work. 


C: Assist with analysis and gain insights from lab data. 
  1. Assess effectiveness of using the combination of GAI and Jupiter notebooks to do guided data exploration. 


notebook and GAI integration

ability to access data files from instructional lab data collection. Does this mean a browser interface with access to local files? 








D: interactive student self-assessments. 
  1. Assess utility of GAI in helping student prioritize where to focus to enhance their mastery of material. 




F: GAI-assisted student assessments and grading


ability to capture GAI sessions, with unique student identifiers
G: Temperature-dependence of learning effectiveness
  1. Assess impact of 'temperature' setting for student learning




H: Ascertain subject-level mastery needed to exploit natural-langauge-driven code development.
         Try out a non-analytic problem and assess the results. 
  1. Use things like modulated-friction example, Modulated Friction example , to help students learn how to assess and refine GAI produced material. 

15 a,b,c
Math 22


numerical solution code
I: incorporate into HW and assessments the ability to perform calculations, as pioneered by Khan Academysame as B

arithmetic capability
J: incorporate course-specific training inputs and give that high weighting


custom training inputs
K: Automation grading and assessments of student competence. 


sequential prompts run open loop, no adjustment
L: dynamic tutoringsame as D

sequential prompts with iterative adjustment 
M: GAI assisted generation and refinement of course instructional and assessment materials- HW, exams, quizzes, etc. 
  1. request a critique of exam questions
  2. request answer key to HW and exam questions




N: Course management- facilitating course selection
  1. upload Canvas course materials and any ppt from previous year, including student CUE evaluation narratives and restrict responses to that material. 


Need to restrict dissemination of uploaded CUE score materials. 
O: Course management- 
  1. restricted-scope utility using uploaded content
  2. integration with grading systems
  3. integration with Canvas
  4. automated sectioning tool
  5. integration with Slack? 
  6. multi-shot 


need to mesh student user account on GAI system with their HUID and course record. 

Instructor-level overview of 


P. Validation and verification tools and methods

  1. Develop methods for assessing and refining GAI-produced code 
  1. Assess level of expertise needed to get credible answers
  2. Conduct iterative-engagement tests 
  1. real-time access to URL's 
  2. provenance tracking
  3. code validation and verification 
  1. refining GAI-produced code 
  1. Assess level of expertise needed to get credible answers
  2. Conduct iterative-engagement tests 


  1. real-time access to URL's 
  2. provenance tracking
  3. code validation and verification 

Deliverableimpact
(1-3 with 5 being high)
1: single course
2: 3-5 courses
3: >5 courses
effort scope
(1-3 with 3 being low)
1: > 1 calendar month
2: ~ 1 calendar month
3: < 1 month
merit

classroom synthesis: Google form gathers student input, sent to  canned prompts and displays on screen, 1-button workflow for instructor313
implement nano-GPT in jupyter notebook 236
implement ability to execute queries on course-specific uploaded materials313
facilitate upload of student sessions as parsable text into grading system326
implement course management tools- grade sheet analysis and interrogation326
incorporate chat functionality into course Slack channel339
new-course development toolkit326
assessment of student preparation, based on instructor expectations326
assessment of student preparation, based only on uploaded course materials313
instructional lab data browsing, tailored to individual courses111
generation of lecture notes from recorded transcripts and video capture of blackbaord? 313
Dynamic HW system, multishot training to guide student interaction313
Assess optimal temperature setting to maximize learning effectiveness (this is an experiment) 313
Assess performance of image capture HW assessment. "work this problem then upload image of your work"326
respondentemail notes
Carlos Arguelescarguelles@fas.harvard.eduHola Chris:

Thanks for sending this email. I am very excited about this new technology. In fact, with Louis Deslauriers, we used some of it in my Physics 15B last semester.
I am currently writing a proposal for the Cottrell Scholars, where this will be featured, and I intend to use them in my incoming classes.
I will look at the links in your email. Have a wonderful summer, cheers,

Carlos




David Malin
using it now in CS50 with custom interface


















...

3. We should identify some undergraduates who can assist with developing course management tools (Brenner and NeesonNesson)

4. Finalize FAQ document (Stubbs)

...