How 10 Lines of AI Code breaks Education

I’ve worked in several domains in my career, Content publishing, Global Operations, Search Engine Development, R&D, Data Science, EdTech and HealthTech.

So figuring out how modern AI could be used in the EdTech industry is of interest to me.

I’m breaking this into a couple of sections

For the non-techies skip over the “Digging Deeper”

The task

We're going to take Great Expectations by Charles Dickens, create a local database with it, pose questions and get answers, and as you see from the headline use 10 lines of code.

Don’t worry if you’re not a coder, just look at how small it is.

The code is tiny and shows how little it’s now taking to integrate something so powerful.

Code :

homework.py

from dotenv import load_dotenv
from pathlib import Path
dotenv_path = Path('settings/.env')
load_dotenv(dotenv_path=dotenv_path)
from llama_index import GPTSimpleVectorIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader('data').load_data()
index = GPTSimpleVectorIndex.from_documents(documents)
while True :
    prompt = input(">>")
    print(index.query(prompt))

That's it!

The code, setup info and test data is available https://github.com/pjaol/homeworker.git

This application will generate a local database from the contents of the data directory, provide you with a prompt and allow you to query it.

Example below (those that know me, know I’m a geek and still use terminals)

Digging deeper

Techie info be here

What we’ve created is often called an Embedding solution, using ChatGPT to turn text into something we can store locally and search against.

What’s stored are vectors rather than specific terms or words, context is “embedded” thus when you search for the term “what are the main goals of the story” you get results about objectives not football scores. Yes I know theres no football in Great Expectations … Yes I mean soccer …

Heres what it looks like at a 10K level

A set of documents gets broken up into tokens, sent to OpenAI’s ChatGPT embeddings interface and the results are stored on your side in a vector db.

When you search for something, that query is sent to OpenAI, again turned into an embedding vector, and now you can query your vector database using similarity queries.

e.g.

Find the nearest vectors to the query vector (hence a similarity search is used)

The results are sent back to OpenAI and reformatted back to text to present to the user.

The benefit is there's an upfront cost to creating your vector db, but then your queries are to a cheaper set of endpoints. So overtime you’ll save some money vs doing a fine tuned model.

This is written about all over the internet with people calling it your

PrivateGPT …. it’s not !!!

You are giving the data to your LLM provider to build the database.

If you are dealing with sensitive data, ensure you have a commercial license giving you confidence on how that data will be treated.

Why does Education need AI?

We can all remember the lines that teachers used say to smart alecks in a classroom.

• You’re not always going to be walking around with a calculator in your pocket (whoops)
  • Why kids had to learn to use log tables
• Where will Jeeves / Yahoo / Google be when you need to know something (with my calculator sir! )
  • Why kids had to remember historical facts
  • Ok I went to school before any of those, so guessing at this one

Slowly it’s come to pass that educational models changed with the times, students are now taught how to use calculators, even graphing calculators!!! How to use the internet and how to do research correctly.

Kids these days are even doing powerpoint presentations, because these are the skills needed for the modern workplace.

Undoubtedly in the next 5 - 10yrs prompt engineering will dominate the workplace, how long will it last? No idea, but education will need to quickly catch up building a new key skill of “Programming the AI”.

The US Education System is in Trouble

There’s no way to avoid saying it

Over the past four decades the US education system has become a victim of politics and forced budget allocations, schools are fighting for resources while money gets directed elsewhere.

High school classrooms are getting larger, as less teachers join or remain in K12

As of 2022 more money is spent on security for students than art classes

• Avg spend of $500 p/a per student on security ($3.2B p/a)
• Avg spend $14 p/a per student on Art studies

As class enrollments are increasing, more students become college bound;

However

• Only 33% of students in the US graduate with a degree in 4yrs
• ~29% graduate with a degree in 6yrs
• 34% drop out carrying debt.

With such high dropout rates, massive debts and a job market that doesn't cover the costs of escalating prices, the current and future generation of students will struggle like never before.

There are clear indicators that students are not prepared for college, or the world, educational-wise, socially, or psychologically, and the return from a college education may not be worth it for many.

A prime example in 2023 a district in Texas discovered at the last minute only 5 students were qualified to graduate from a class of 33

• https://www.nbcnews.com/news/us-news/texas-school-district-commencement-called-only-five-33-prospective-stu-rcna86444

An onus placed on both students and school counselors.

If the counselors whose job is to help students remain on track are struggling, how can students succeed?

The US education system is at a breaking point.

Can AI help?

Yes - in many ways

Todays solutions for Educational Gaps are Intervention based, and measured in RTI; response to intervention. It's generally broken into tiers

• Tier 1 - whole class instruction
  • Teachers direct and entire class, sometimes break students working into groups
  • Difficult to provide individual attention
  • ~80-90% Success rate
• Tier 2 - small group instruction
  • Students groups based on skill levels
  • Meet multiple times a week
  • 10-15% Success rate
• Tier 3 - individual instruction
  • Remedial classes
  • Smaller groups
  • 1-5% Success rate

As you can imagine, this is a responsive post fact intervention, often creating a stigmatizing event for a student.

Students failing to stay on track, descend from one tier to the next.

What if - the student was able to receive personalized support at the time the are studying rather than weeks or months later after they've failed and the class has moved on?

Adaptive Teaching

For years multiple versions of the same curriculum has been developed with the idea of catering to lower and mid range students. The classroom has struggled with teaching more than one curriculum due to resource constraints.

However today, AI can be used to create personalized and adaptive lessons on the fly. Even Shakespeare can be simplified with a slider on an iPad to match the reading and comprehension abilities of the individual student.

Providing students with a lesson plan that they can make their own, adapting to their needs while giving them the ability to reach for more.

Actionable Real Time Feedback

For the teachers AI can also provides them with a deeper understanding of the students needs by analyzing how each student is learning, identifying where they have problems and even recommending how to help that student improve. Again while the student is in the process of learning. Without the need for a test, or the need for a teacher to grade results.

The key to AI in Education, is to provide personalized, adaptive scale to meet students where they are in their educational journey.

While also providing Teachers a deeper understanding of where there students are and what help they need.

Again it should never be considered a replacement for Teachers, but as a tool to help overcome the hurdles of socioeconomic statuses that students deal with.

As well as making providing support to an already stretched thin set of resources. 

Want to learn more about what The VGER Group can do for Education?