Suppose you have a transformer that handles tokens of dimension Dmodel.D_{model}. And suppose it has ten tokens in the input. The residual stream that’s updated
Category: Ideas
Alignment as a Vocation?
I just re-read Max Weber’s essay “Science as a Vocation” in the context of thinking about human and machine intelligence alignment. In particular, I’m thinking
Learning from Every Keystroke
I suspect that this is a naive question already thought through by people much smarter than I, but here goes. We’ve heard some concerns expressed
Teaching with My Whole Palette
All told, I have probably spent 20 full person-years in college and university classrooms. I’ve taught courses in sociology, public policy, information science, design, philosophy,
Platforms for Pedagogy I
First Installment in a Month-Long Project “Information Systems for Pedagogical Productivity” Are you like me? A college or university instructor whose computer storage is littered
Entrance vs Exit Exams
My students often care about the grade they earn in my course because they believe it will influence subsequent opportunities. And so they care about
