Building a product using GenAI

This week we announced a new product called Advocacy Boost.

It’s not a ground breaking product, it's not a money maker, if it breaks even I’ll be happy, it was built out of frustration.

Advocacy Boost is a product that solved a problem that we faced, how to collect customer testimonials.

Every tool that we looked at had hurdles

• Required your customers to
  • create an account
  • download software
  • navigate a new product
  • deal with marketing
  • figure out how to share the video they created somehow
• Most platforms disable downloads
  • They force you to pay subscriptions to host your video
  • They watermark your content
  • They limit your editing abilities

It’s essentially a blackmail business model, where even if you pay to upgrade and gain some freedoms, you still have to get your customer to jump through hoops to record the testimonial.

So we decided to solve it, in 2024 recording a video, sharing a desktop, storing it through a browser isn’t that hard. Except we haven’t done that before, our expertise is in AI, Data, Architecture, and Operations.

Could we use AI to help us build it?

The answer is “yes, but…”

The objective, a web platform that allowed companies to invite customers to record a video testimonial.

The tools, ChatGPT4, Python, Javascript, AWS S3, Heroku, Stripe, Co-Pilot, Musho-AI + Figma

Initially we were using some open source frameworks for the video recording, until they broke, and we quickly discovered WebRTC is really finicky, and next to near impossible to debug. That's when we pulled in some GenAI.

With a little work we got ChatGPT to build a JS library that did exactly what we needed.

We provided the requirements, tested the output, refined the requirements, identified the bugs and iterated.

Within 2 days we have a JS library that worked~ish (yeah Safari is a nightmare with WebRTC)

Next we wanted to store data in AWS and use a browser so we're not having to build a server component, or mount a file systems to store a video file.. That took 2 days, should have taken one but the python library we were using has a 3yr old bug that we had to fix. :-)

https://stackoverflow.com/a/78054212/1186931

Next was Stripe, turns out there are no integrations off the shelf for Stripe and Python / Flask, we had to build our own shopping cart. That’s what we get for using Python. Add 4 days for cart, checkout, webhooks, testing etc…

Lastly UX, our weakest link, turns out there’s some amazing Text to UX tools like Musho-AI https://musho.ai/ that just builds you a UX. It’s in Figma which means you need a young priest and an old priest to get it into usable HTML format, but we got there.

Overall we went from 0 - 95% in the space of 2 weeks with 1 person,

A standard web development team UI/UX, QA, 2 x Dev, PM running scrum & sprints would take about 8-12 weeks.

Is building a product with AI possible?

yes.

But you need the following skills :

• Engineering experience in the languages used.
• Ability to write requirements, both high level and low level when it doesn’t work
• Ability to test, refine, research ways to solve the problem
• Patience - a lot of it

A good engineer can put GenAI to work often doing the mundane tasks.
A great engineer can put GenAI to work by identifying problems for it to solve. 

The process

There are several projects out there that can build small single - few file based apps, ChatDev is one of more popular one. They're not going to be able to help with the complexity of something like this.

Some stats on Advocacy Boost

This is honestly an employee sitting 9-5 everyday at a screen asking ChatGPT to build another requirement, testing it, feeding bugs back in to ChatGPT and iterating.

Some tips:

• Conversation AI's rely upon chat memory to insert context 
• Occasionally you have to resupply the context by copy and pasting previous code into the conversation
• Responses can often break code by summarizing changes
  • You will need to ask for the full code multiple times
• Commit code continuously
  • You will need to compare and contrast the code
• Know when to stop and find an alternative solution
  • GenAI tends to be stubborn, and provide the same approach time and time again
  • At some point you need to identify it's not working and find an alternative approach and direct the conversation to a different solution.
• GenAI wants to say Yes
  • This is a tough one, RLHF reinforced learning from human feedback tends to make your AI tools want to say 'Yes', and this can lead you down a rabbit hole. 
  • You need to know if what your asking is possible
  • You might need to hint and push your conversation towards examples and solutions

Where does that leave us today?

Overall GenAI built 70% of the product, with 80% requiring human correction.

It is far easier and faster to correct than it is to create.

Is the product perfect, no, it’s a viable product, but not minimal it has pretty much what we needed, and is better than almost any first releases I’ve seen. I'm biased but I’m still shocked.

People have been asking will AI replace employees, maybe, but more likely it impacts companies in ways they haven't thought about yet..

• Companies have been foolishly trying to hire 10X engineers, where output is measured in volume.
• Engineers by mastering tools like GenAI can solve problems like never before, delivering 10X in value.
• GenAI will lower the barriers to build products that previously only large companies were seen as having the resources to do.
• With GenAI companies will no longer be able to rely on locked-in motes or IP as a business model, any decent engineering team can build an alternative in almost no time.

We’re not a WebRTC company but we built in our spare time a go to market product in an area we don’t know, with 1/5th of the resources in 1/4 of the time it should take.

We just knew what we wanted, and used tools to create it.