Related to tutor, I've been playing a "game" lately with ChatGPT-4. I will give it a prompt resembling "You are a tutor. You will guide me to understand a topic, without giving me direct answers. I will guess how something works, and you'll give small hints for what I should explore further, or reconsider. The concept is: electrorheological materials."
Even without AI, this is one of my favorite things to do, and easiest way to learn, where I guess, with as much detail as I can, and then dive into reference material to see how close I was. I've had a few car rides now with the iOS app's conversation mode doing this. AI is the clearly the future of education, in my opinion.
A similar, more passive approach is to tell the AI to conjure 3 or 4 personas representing the main points of view about a particular topic (i.e. "universal basic income"). Each persona is an eloquent expert in the field with a strong opinion aligned with each of the PoVs. Then I ask them to debate the topic. Really insightful stuff.
For the ultimate version of this, once locally trained and running LLMs becomes a bit more feasible (meaning, closer to GPT4), it would be a fun exercise to train various models with personas and let them hash it out until they all agree on something.
One model could be trained on HN, another on a specific Facebook user group (like "Moms against video games"), another on Twitter on so on, then you let them come together until they've reached consensus (or not).
What's more likely, that you're commenting on the original HN or that you're entire life and experiences are a blip in the training data for this project?
If full Boltzmann brains with experiences can pop out of random space as the age of the universe tends towards infinity, why not from ML training?
I don’t think they will come to a consensus. These personas aren’t actually interested in arriving at some fundamental truth, or having their mind changed. They’re interested in defending a specific position.
It might be better if you can spin up separate LLM's for each one, each with its own unique context or even fine tuning (or, at least, start each one's response with a fresh session using its own individually-tailored prompt and context).
Interesting suggestions. I will give them a try later.
On a side note, lately I enjoyed some interactions with ChatGPT more than social interactions. Given the right circumstances, CGPT can be very interesting to interact with.
Oh thanks! I actually never considered this a direct application. I suspect I can shorten my prompt considerably.
But, I'm not convinced about "common". I've had several highly educated people, from top tier universities, outright offended that I would guess about anything, rather than go straight to the literature. But, they were also no fun.
I have encountered people that claim to use Socratic and/or Zeteticism as alternatives to scientific method to explain why ghosts are real or the Earth is flat.
So it's possible you mentioning it got their haunches up
Professors in law school love to use the Socratic method. Most classes are centered around the professor asking students about cases from the reading. The student describes the case and answers follow-up questions about it from the professor.
That's a useful distinction in these discussions. Learning methods that work for motivated individual learners often cannot be applied to large groups of students who are required to study a subject.
I personally think that institutionalized education will have to adopt and adapt to AI, but it won't be easy.
It's amazing you can do this. My wife cannot even get Android auto to transcribe addresses correctly--we need to "speak" an address when the car is moving. It works a bit better for my "strong male" voice, but still quite a frustrating experience.
It usually transcribes correctly, but if you pause at all, it'll end the dictation. It requires an unnatural speaking style, without many pauses. I'm really excited for the new conversation mode, that doesn't rely on the phones dictation [1].
Ultimately history books are written by the winners. So the question is not whether ai will be the future, but which ai will be the future. From that perspective, it’s just more of the same old same old top down.
This article is avoiding the elephant in the room. From talking to students it is quite clear to me that the main student use case for AI is as a homework assistant. You tell ChatGPT your homework question and ask what's the answer.
In some cases ChatGPT will simply give you the perfect answer and there's nothing more you need to do. In some cases ChatGPT will give you an incomplete answer and you need to finish it up yourself. Often here asking some followup related questions will be quite useful. In some cases ChatGPT will give you a subtly wrong answer and you need to figure that out.
I think the toughest ethical situations arise when ChatGPT can give you a perfect answer to your homework problem. Is it unethical for a student to use ChatGPT in this case? Does superficially rewriting the answer to put it in your own words make it ethical? Is it appropriate for professors to give out A's to the students who cut and paste from GPT4, while giving lower grades to students who don't realize that such a tool is available, or who won't pay $20/month for the best homework tool?
American education prior to GPT had increasingly taken a turn in emphasizing homework over testing. My former high school, for instance, reduced the impact of testing to ~15% of total grade and made homework/classwork the vast majority. This steals time from the rest of kids lives, makes high school performance more a function of “time spent” than understanding, and completely misses the mark of what the goal of schooling ought to be in my view.
I think GPT4 and the like spell the end of homework and that is a good thing.
Your personal experience notwithstanding, this is not the broad trend in education.
Mileage will vary between states, districts, and even schools, but the big current trend is towards “standards based grading”. This means grades are based entirely on test, projects, etc. Individual work meant to assess competence. Homework, practice, group projects, class participation, etc count for token credit if at all.
This is a huge, recent shift in direction, and probably came after your time in HS. I don’t have data for this, but expect public districts to lag charters, rich districts to lag poor districts (la cañada USD has no incentive to rock the boat), and privates to be the Wild West.
But anyway homework has been on the way out for a little while now. This is great! Most homework is worthless.
Math says otherwise, my mastery of math in college came through hours of slogging through problem sets (which the answers are often provided for!) until I understood all the many ways to apply a concept.
Prior to college I had skated by w/o doing homework (I understood what was said in class, so why bother?) and it bit me really hard. Understanding what it said during a lecture is one thing, internalizing that knowledge, and knowing how to apply it, means actually putting in the time and effort.
> This is a huge, recent shift in direction, and probably came after your time in HS.
I am probably younger than you think and 'standards-based' approaches were a big fad at the time. Standards based grading in highschool ("SWBAT"-style) absolutely does not mean grades based entirely on test, projects, etc, and in fact generally translated to heavy weighting towards random classwork worksheets.
I would like to point out that US education isn't exactly a paragon of quality compared to other countries, where this approach also (maybe coincidentally) isn't practiced.
It will force a measure of actual aptitude, with proctored exams, with some lady with a ruler slapping your desk if you look at your neighbor. What's old is new!
Which is almost impossible. Exams just measure your aptitude at passing exams, with the hope that they somehow don't generalize too badly. If we had a way to measure "actual" aptitude, job interviews would be very different. But we don't.
The ideal school system is one where students want to learn, as opposed to one where students want to make good grades. Unfortunately that's hard to get, but it should not be forgotten.
I think tests are probably one of the closest things we can get to. If I ask you a question about the thing you claim aptitude about, and you cannot answer it but we are claiming that doesn't measure aptitude, then I think we are getting into the weeds of what semantically aptitude is.
> If I ask you a question about the thing you claim aptitude about, and you cannot answer it
That's the trivial part, nobody denies this. The harder part is the reverse: if you ask me a question about the thing I claim aptitude about, and I can answer it, does it prove I have aptitude for that thing, or does it prove I have the aptitude to answer this question?
If I learn by heart the 20 math problems you gave me to practice for the test, and then you test me on a similar problem, then I will be able to "do stuff" that is not completely meaningless by just mimicking the previous problems. Students often do that, and get the average even though they got the wrong answer. Many times, looking at their errors, you can be pretty sure that they just don't know what they are doing. But teachers cannot give none or all the points (or all minus 1 if there was a small mistake), it has to be a gradient. In my school system, students who get just the average grade for math problems have absolutely no clue what they are doing. If they did they would get one of the best grades. Still they pass.
Same goes for tech interviews: people learn algorithms and keywords and problems by heart just for the interview. Doesn't mean they know how to write good software, just that they learnt those things by heart.
> Same goes for tech interviews: people learn algorithms and keywords and problems by heart just for the interview. Doesn't mean they know how to write good software, just that they learnt those things by heart.
For actually hard questions, I think this happens a lot less than is claimed.
I would be astonished if someone without physics knowledge could 'learn by heart' and get anywhere close to a passing score on one of my college physics tests.
Maybe I'm just rabidly pro-test, but I think a lot of the anti-test sentiment comes from... people having negative experiences with testing in their own lives, than an actual indictment on its utility. The obvious explanation for why people hate leetcode is because it gates them from $200k+ jobs, not because of some high-minded moral qualms with tests.
> I would be astonished if someone without physics knowledge could 'learn by heart' and get anywhere close to a passing score on one of my college physics tests.
Again, I am not saying specifically that "someone without physics knowledge" can get a passing score on college physics tests. It's a bit more nuanced than that, and it's difficult to compare because I did not study in the US. And passing a test doesn't mean nothing (if anything, it shows that you are capable of passing the test).
But I have the feeling that I myself was pretty good at optimizing for the grades, even if retrospectively, that was not always the most constructive learning experience. As an engineering undergrad, I studied one year abroad, and there I could literally take twice as many classes as "local" students and get the best grades. First I thought "they sucked", and then I realized that the students there did not care so much about the grade: it was easy to pass if you genuinely put the work into it, and it was not uncommon to ask to redo a class on a voluntary basis. They were studying to learn, not to get the best grade. And sometimes it implied going slightly aside from the main course topic out of interest (instead I was doing exactly what was needed to get the best grades and the most classes).
Not to say I'm particularly bad though I had very good grades, or that they were particularly good with bad grades. Of course they had good and bad students, too. It was just interesting to see that it worked completely differently.
Then it made me think about school/high school, were I always had excellent grades: how did it make the other students feel like? Grades give some kind of ranking, which can feel like a competition. And I remember clearly (and saw it again later as a supply teacher) that those struggling to get the average grade would often optimize for that: "I don't have time to understand how to do it, I need to get the average". A bit like software engineers don't have time to "do it right" and write shitty hacks instead: it's not that it doesn't work at all... but that is certainly not the most constructive way. And definitely what I saw as a supply teacher was that some of those students were simply wasting their time in the maths classes, but in a very stressful way because they had to find tricks to get the average without actually understanding the maths.
> The obvious explanation for why people hate leetcode is because it gates them from $200k+ jobs
Admittedly I haven't tried that. But regarding grades, I am most critical of the grading system in school/high school, and I had excellent grades there. Then at university level, this exchange year showed me that excellent engineers came out of a school where grades did not matter so much, so it is possible.
I guess my point about leetcode is that I have seen software engineers who excel at those, but write unreadable and unmaintainable code. Which tells me that leetcode doesn't measure the capacity at being a good software engineer in general.
There are physical things that can be done. Currently, people get around video proctors by taping things to their monitor, then the camera can't see. A lockdown browser would assume the person owned exactly one computing device.
> I think GPT4 and the like spell the end of homework and that is a good thing.
In my kids' case it hasn't mean the end of homework, rather it created a new policy that only authorized devices (school supplied Chromebooks) can be used while in school. Completing schoolwork on your own device even outside of school hours is also frowned upon.
They say the change is about "network management" but parents and students both know the real reason.
I mean for most large businesses you will be provided with a company issued device and taking company work of said device may be grounds for termination.
In business it's generally to prevent the leaking of trade secrets, but there are more than one cases where an employee was allowing a surrogate to connect to their device and do the work as a form of unauthorized subcontractor.
> I think the toughest ethical situations arise when ChatGPT can give you a perfect answer to your homework problem. Is it unethical for a student to use ChatGPT in this case? Does superficially rewriting the answer to put it in your own words make it ethical? Is it appropriate for professors to give out A's to the students who cut and paste from GPT4, while giving lower grades to students who don't realize that such a tool is available, or who won't pay $20/month for the best homework tool?
How are any of these tough ethical situations? The obvious answers are "yes", "no", and "that's moot." (Moot because the ethical onus is on the student, not the professor. It's not the professor's fault that the student is benefiting from unethical behavior.)
People get assistance in homework/problem sets/take home tests using lots of different resources and tools. What’s allowed and not allowed varies based in the “rules” and common conventions.
GP's framing assumes that the homework is being graded to assess competency: "Is it appropriate for professors to give out A's..." If that's not the case, no ethical questions are involved in using Chat GPT to complete your homework, although it would seem to be a pointless exercise, since you are neither getting practice nor cheating the assessment mechanism.
Well, let's take the first case in more detail. Is it unethical to talk to ChatGPT about your homework in the first place? I don't think that seems right. Many times it will just be a helpful advisor and you will really learn a lot from its responses.
So, you ask ChatGPT about your homework question, and sometimes it turns out it just gives you the perfect answer. What now? You can't un-see the answer. If not even rewriting the answer makes it ethical, are you just trapped and there is no ethical way to handle this situation? You could conclude it's unethical to use ChatGPT for anything homework-related, that is at least consistent.
Let's consider the original framing: "You tell ChatGPT your homework question and ask what's the answer." You aren't asking for help with the homework in this framing, you're just asking for the answers so you don't have to actually do it. I don't think this scenario raises any new, difficult ethical questions; it's exactly equivalent to copying the homework of a friend.
Personally I think it just comes down to the student cheating themselves. If they're in a job interview and don't have ChatGPT handy AND they've been using ChatGPT to get answers directly instead of asking how to arrive at the answer, I think they're going to have a hard time getting that job.
However, if they always have ChatGPT available and can always get the right answers (either by luck, prompt engineering skills, or an ability to determine if an answer is correct), I don't see them having a hard time competing in the workplace at all for most commonly solved problems.
Hopefully the teacher would do a sufficient job teaching such that if the non-ChatGPT-using student puts forth adequate effort, they'll be able to get the answers as well, but maybe not as quickly as those using ChatGPT.
And then the kid who refrained from cheating themselves by using ChatGPT arrives at the job interview, gets asked to answer a bunch of difficult questions with the help of ChatGPT, and fails because of lack of modern computer skills. :-)
Yeah, if you take school as an adversarial system, you might see LLMs as a threat, but Harvard doesn’t see it that way. To them, college is a chance to grow your understanding of a wide variety of topics, and to gain expertise in one (or two) topics.
Nobody thinking like that would see LLMs as a risk.
To me this seems a very similar argument to when Google/Wikipedia were gaining relevance.
Many IT professionals agree that most of their job is knowing how to Google things. Sure they need to know how to read manuals, forum posts, find product numbers/specs., simplify the information for the end user, etc., but at the end of the day the skill is mostly knowing how to properly and effectively leverage an external source of knowledge/wisdom/intelligence in a occupational domain.
ChatGPT is just Google on steroids in this respect.
> ChatGPT is just Google on steroids in this respect.
Which doesn't mean it is not creating a new problem, does it? "A car is just a skateboard on steroids", still we created many laws around the use of cars.
> Is it unethical for a student to use ChatGPT in this case?
I think this is the wrong question to ask. The real question is: "is it smart for a student to use ChatGPT instead of doing their homework?". And probably it is completely stupid. School is not an adversary, school is here to teach. If students manage to trick the system to get good grades while staying stupid, it doesn't really hurt anyone other than themselves.
> School is not an adversary, school is here to teach.
This is debatable, at least in the US. In many cases, higher ed is here to offer a credential and network regardless of true aptitude since many students will simply not use the majority of what they are supposed to be learning in their career.
Put another way, students were already using things like ratemyprofessor to deliberately tailor their courses for easier grades back when I was in school, plus memorizing older exams rather than studying the core material in detail, or just straight up paying someone else to write papers for you. AI is an extension of this, not something new.
Some students will always try to cheat, that is inevitable.
But saying that whether schools are for teaching is "debatable," while a popular meme, is a false and harmful assertion.
Regardless of what you or any of us consider to be "proper education" or "good teaching," even if higher ed really is just about getting certified for jobs, the simple fact is that the purpose of school at any level is to learn various skills and facts, and any form of direct-to-the-answer interferes with that.
I saw this a lot when GPT3 first came out, many people on HN and elsewhere decried the existence of book reports and essays, rejoicing that such "worthless" tasks would go the way of the dinosaur. However, all who say such things miss the crucial insight: the point of the book report is to learn and display reading comprehension. Similarly, essay writing is only about learning to communicate with essays specifically in the first one or two classes you encounter such tasks. After that, essays are used to communicate your command of the material for subjects that cannot be easily broken down into multiple-choice or short-form exams.
Allowing students to use AI tools to do their homework for them is a massive step backwards, and an unimaginable risk to the future of our society. If we allow our children to use AI to do all their work for them, especially during crucial reading comprehension years, they will arrive at adulthood unable to process anything more complex than a menu.
And relying on AI to summarize and communicate information that would otherwise be written into a document of some kind is an even bigger risk - then the controllers of the AI would essentially control how we process information in general, because no one would be able to do it themselves.
21% of adult Americans are already illiterate, pre-AI. 54% read below a 6th-grade level. What kind of world are we building if we push those numbers even higher?
The hard part is to get students to understand why the exercise is useful. It is easy to lose one's time writing an essay. But if you lose your time, it means you are doing it wrong; better not to do it at all.
This assumes the schooling is not functioning largely as a signaling device, which at Harvard it may be.
I’m not saying students should use ChatGPT on their homework, but I would understand why a student would do so if it allows them to spend more time on side projects, networking, learning about other topics, etc.
Tracking is prevalent everywhere in the American education system and means that this argument is wrong. If you are getting downtracked because everyone else is cheating with ChatGPT, that's bad.
Sure but as I said above, instead of fixing the ChatGPT problem that happens because of the broken education system, maybe there is an opportunity to improve the education system.
Well, tracking is good if it helps the student learn. But if it is commonly admitted that the students need to trick the system to get higher ed and will then get better jobs because of that, maybe there is something to rethink about how higher ed is valued.
Schools haven't caught up, but it will be possible to adapt to new forms of assessment that test the merits of this new type of human who probably has a personal assistant who knows almost everything with them at all times. It should probably look pretty different to how we've done things up till now.
The average person/studen can't install a llama 2 model (70 billion parameter) that can compete with ChatGPT-4 (the paid version), considering most students have notebooks, and lack top tier GPUs. Most would have to go to a "free" cloud providers.
ChatGPT-3.5 is also free, and compares favorably against the lower parameter llama 2 models. But, llama 1, lower parameter/heavily quantized llama 2, and ChatGPT-3.5, are not practically comparable to ChatGPT-4, for most tasks [1]. I haven't found a use for those, in this sort of context, except for increasing my blood pressure, with the time wasted being my justification for paying $.70/day.
Nonsense, for most tasks even 7B parameters is enough to replace chatgpt for most tasks. Arguably 7B is even overblown. The goal is not "human-like" but "profitable"
I feel like the one thing that is systematically missed about generative AI is the scale.
Yes, one could ask their older brother to help. But not everyone, not for every single homework.
Yes, one could manually find information about a target and forge a phishing e-mail. But generative AI make it much faster/easier.
The scale means that it is different. Saying "it is not a new issue" is a bit like saying (for the lack of a better example): "well children could always throw stones at each other, letting them have machine guns is not a new issue".
I had a grad school class that involved some math (encryption algorithms). To help understand some concepts I would check in with Chat GPT and use it as another resource along with the text, lectures, and youtube. At one point I used it to check some of the math and I noticed was that it simply did not do math well. It seemed to understand the steps (or most), but the numbers would just be wrong. So if any students out there just blindly pasted the results without checking, they would be wrong.
> In some cases ChatGPT will give you a subtly wrong answer and you need to figure that out.
The interesting question here is whether figuring out whether an answer is subtly wrong is more efficient/effective than writing the answer yourself. It's probably more fun though.
Why have school at all? Teach kids to read and how to use the internet. Give them a coach that trains them on how to achieve what they want using AI. And, when it's necessary, interview them for competency. I can't see how this isn't the future.
People (especially kids) usually lack the personal will power to plough through a subject matter on their own. Having a real human placing demands and expecting results does wonders. Being part of a class taking the same subject also helps with motivation. Professors and other kids don't need to be prompted to talk to you, so they are not so easy to avoid as a chat bot.
You can outsource work but can't outsource wishing. You got to know how to demand and what to reject, or else, if you try to automate or outsource that, the other people and AIs won't necessarily work in your best interest. They follow their own interests, you got to do it yourself.
Not knowing that something exists is a sure way of never demanding it. Having an education is even more useful today when all you need to do is knowing how to ask.
Can we remove "Harvard:" from the title? It is editorializing contrary to HN guidelines and this is not an official position of Harvard but rather an article published by Harvard Business Publishing (not written by Harvard faculty).
For the last few months Ethan Mollick has been writing some great articles about how to best use Generative AI in his blog: https://www.oneusefulthing.org/
I think it could really use some prompt or workflow examples. It mentions pressing the LLM for more information and to explain itself, which could be demonstrated using some prompts.
edit: Oh my mistake -- I didn't realize that the links in the article lead into more articles that do have examples.
The author Ethan Mollick has been doing some of the most in-depth testing of LLMs that I've seen (on Twitter). I highly recommend following as he's found quirks and use cases with LLMs earlier than a lot of more technical prompt engineers.
I can see that one of the authors of the guide is Ethan Mollick. He has a free substack with more AI advice. You may find his own guide very useful (https://www.oneusefulthing.org/p/how-to-use-ai-to-do-stuff-a...). This is aimed at a more general audience.
One of the most important lessons I got from high school came from my history teacher: instead of testing that we had learned the book by heart and were able to "copy-paste" the right parts into our test, she was looking for critical thinking.
In a typical test, she would give us all kind of articles (with the source: date, author, journal, country, etc) and ask us to analyze it. Not only it required us to understand the historical context ("the author is talking about this event that happened at this place for this reason"), but we had to show critical thinking: "he comes from this country and has this religion, so I would assume that he has a bias towards this", etc. To the point where we could even choose which questions we wanted to answer in the test. "It's okay to have completely forgotten about this event, just pick a question about some historical context you know". She was not testing our capacity at learning everything by heart (of course we had to learn something, otherwise we could not answer any question at all).
It took her time to establish that. Many students hated her, because they would usually easily make good grades by "learning the book by heart", and they were failing here. But at least for me, when I started understanding how it worked I also started loving this class. Somehow I did not feel like another pointless class where I would optimize my grades: I was actually improving my critical thinking. I was getting something out of this class other than good grades. Reading my history book now was not about learning facts, but trying to understand the context through the lens of the author.
I think that every teacher should strive to give that kind of feeling to their students. Then ChatGPT would not be a problem anymore: it becomes clear that it is not trustworthy. If some like to use it as a tool (e.g. because they need ChatGPT to give them keywords that they can use in further research), that's fine. Those who take the output of ChatGPT for granted have just missed the point, they may as well do nothing.
Also... how does it have any value then? Either I should ignore it (in which case ChatGPT was useless) or I already know it from a trustworthy source (in which case ChatGPT was useless).
From my experience, these types of prompts are only good for three or four responses/follow ups, then it somewhat quickly "degrades" to more standard behavior, forgetting the detailed instructions.
I think it's possible in a prompt for the LLM to remind itself of it's purpose in its own responses, creating a sort of flywheel effect. If you use a longer context window model (16k, 32k, 100k) this shouldn't be as much of a problem.
It depends on the total context of the conversation. As the number of tokens grow, it starts to lose track. Usually it's noticeable and you can remind it of what it's missing.
Even without AI, this is one of my favorite things to do, and easiest way to learn, where I guess, with as much detail as I can, and then dive into reference material to see how close I was. I've had a few car rides now with the iOS app's conversation mode doing this. AI is the clearly the future of education, in my opinion.