I’ve worked in several domains in my career, Content publishing, Global Operations, Search Engine Development, R&D, Data Science, EdTech and HealthTech.
So figuring out how modern AI could be used in the EdTech industry is of interest to me.
I’m breaking this into a couple of sections
For the non-techies skip over the “Digging Deeper”
We're going to take Great Expectations by Charles Dickens, create a local database with it, pose questions and get answers, and as you see from the headline use 10 lines of code.
Don’t worry if you’re not a coder, just look at how small it is.
The code is tiny and shows how little it’s now taking to integrate something so powerful.
Code :
homework.py
from dotenv import load_dotenv
from pathlib import Path
dotenv_path = Path('settings/.env')
load_dotenv(dotenv_path=dotenv_path)
from llama_index import GPTSimpleVectorIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader('data').load_data()
index = GPTSimpleVectorIndex.from_documents(documents)
while True :
prompt = input(">>")
print(index.query(prompt))
That's it!
The code, setup info and test data is available https://github.com/pjaol/homeworker.git
This application will generate a local database from the contents of the data directory, provide you with a prompt and allow you to query it.
Example below (those that know me, know I’m a geek and still use terminals)
Techie info be here
What we’ve created is often called an Embedding solution, using ChatGPT to turn text into something we can store locally and search against.
What’s stored are vectors rather than specific terms or words, context is “embedded” thus when you search for the term “what are the main goals of the story” you get results about objectives not football scores. Yes I know theres no football in Great Expectations … Yes I mean soccer …
Heres what it looks like at a 10K level
A set of documents gets broken up into tokens, sent to OpenAI’s ChatGPT embeddings interface and the results are stored on your side in a vector db.
When you search for something, that query is sent to OpenAI, again turned into an embedding vector, and now you can query your vector database using similarity queries.
e.g.
Find the nearest vectors to the query vector (hence a similarity search is used)
The results are sent back to OpenAI and reformatted back to text to present to the user.
The benefit is there's an upfront cost to creating your vector db, but then your queries are to a cheaper set of endpoints. So overtime you’ll save some money vs doing a fine tuned model.
This is written about all over the internet with people calling it your
PrivateGPT …. it’s not !!!
You are giving the data to your LLM provider to build the database.
If you are dealing with sensitive data, ensure you have a commercial license giving you confidence on how that data will be treated.
We can all remember the lines that teachers used say to smart alecks in a classroom.
Slowly it’s come to pass that educational models changed with the times, students are now taught how to use calculators, even graphing calculators!!! How to use the internet and how to do research correctly.
Kids these days are even doing powerpoint presentations, because these are the skills needed for the modern workplace.
Undoubtedly in the next 5 - 10yrs prompt engineering will dominate the workplace, how long will it last? No idea, but education will need to quickly catch up building a new key skill of “Programming the AI”.
There’s no way to avoid saying it
Over the past four decades the US education system has become a victim of politics and forced budget allocations, schools are fighting for resources while money gets directed elsewhere.
High school classrooms are getting larger, as less teachers join or remain in K12
As of 2022 more money is spent on security for students than art classes
As class enrollments are increasing, more students become college bound;
However
With such high dropout rates, massive debts and a job market that doesn't cover the costs of escalating prices, the current and future generation of students will struggle like never before.
There are clear indicators that students are not prepared for college, or the world, educational-wise, socially, or psychologically, and the return from a college education may not be worth it for many.
A prime example in 2023 a district in Texas discovered at the last minute only 5 students were qualified to graduate from a class of 33
An onus placed on both students and school counselors.
If the counselors whose job is to help students remain on track are struggling, how can students succeed?
The US education system is at a breaking point.
Yes - in many ways
Todays solutions for Educational Gaps are Intervention based, and measured in RTI; response to intervention. It's generally broken into tiers
As you can imagine, this is a responsive post fact intervention, often creating a stigmatizing event for a student.
Students failing to stay on track, descend from one tier to the next.
What if - the student was able to receive personalized support at the time the are studying rather than weeks or months later after they've failed and the class has moved on?
Adaptive Teaching
For years multiple versions of the same curriculum has been developed with the idea of catering to lower and mid range students. The classroom has struggled with teaching more than one curriculum due to resource constraints.
However today, AI can be used to create personalized and adaptive lessons on the fly. Even Shakespeare can be simplified with a slider on an iPad to match the reading and comprehension abilities of the individual student.
Providing students with a lesson plan that they can make their own, adapting to their needs while giving them the ability to reach for more.
Actionable Real Time Feedback
For the teachers AI can also provides them with a deeper understanding of the students needs by analyzing how each student is learning, identifying where they have problems and even recommending how to help that student improve. Again while the student is in the process of learning. Without the need for a test, or the need for a teacher to grade results.
The key to AI in Education, is to provide personalized, adaptive scale to meet students where they are in their educational journey.
While also providing Teachers a deeper understanding of where there students are and what help they need.
Again it should never be considered a replacement for Teachers, but as a tool to help overcome the hurdles of socioeconomic statuses that students deal with.
As well as making providing support to an already stretched thin set of resources.
Want to learn more about what The VGER Group can do for Education?