Cypress Tips June 2025
RAG for Cypress questions 🤖
Retrieval-augmented generation
Imagine yourself coding a Cypress test. In your code editor you prepare a test; you enter its title and type the comments planning the test steps.
it('changes the text after clicking the button', () => {
cy.visit('index.html')
// there is an element with id "message" that contains some text
// after clicking on the button with id "button"
// the text changes to something else
})You can see the initial test running in Cypress
Let’s ask AI to write the test for us. I will use GPT-4o model included with GitHub Copilot to generate the solution
Hmm, that is not a good answer. Like at all! Using older models like “Gemini 2.0 Flash” and “cursor-small” produces similar results.
Let’s switch the model to Claude Sonnet 3.7 and try again.
Trust but verify
Ok, this is slightly better answer with one subtle bug. The initial message text is stored and loaded from an alias “originalText”. Cypress re-evaluates the aliases when you retrieve them - thus the original text and the updated text after clicking the button … will always be the same!
We can fix the solution by adding the “type: static” to the alias
cy.get('#message')
.invoke('text')
.as('originalText', { type: 'static' })More and more data
Let’s try using Cursor. It “vacuums” all the data in the local project, reading the index.html file and grabbing the actual HTML text before and after the click.
The above answer fundamentally misunderstands the task. I am not looking for specific text assertions, since in most cases they would not be known. The task was to check if the unknown text changes!
Ok, why is AI suggesting wrong answers? Older / smaller models generate simpler incorrect answers, while newer larger models throw everything at the problem and get confused. Can we do better?
Aside: I strongly believe the rush to produce MCP servers that extend AI chat sessions with more skills is misguided, since it leads AI on wild goose chases.
Give AI a relevant example
Here is a better way to answer Cypress-related question. If you were a human familiar with this newsletter, before answering the “implement the test following the comments” prompt, you would open your browser, go to https://cypress.tips/search and search for an existing Cypress test example similar to the task at hand:
There is already a similar example from at glebbahmutov.com/cypress-examples!
Nice. Let’s grab the Markdown of this found recipe and copy to clipboard
Go back to the VSCode Copilot or Cursor chat and include the raw Markdown with your original query:
Can an older / smaller LLM generate a good answer now? Yes, it can!
Beautiful. Including relevant Cypress example with the prompt lets even smaller / faster / cheaper LLM produce good tests. Even accounting for more tokens, the larger prompt can still come out cheaper because we can use a cheaper model to run it, not even counting the human time spent on reviewing / fixing the answer.
The newest LLMs scrape a lot of data, but most data on the Internet is low quality. Using tested, verified examples from my blog posts, cypress-examples site, online lessons is THE best way to generate high-quality answers in your specific situations.
Stay tuned for more, as I show how to automate the manual “search” step using text vector embeddings. I am working on blog posts and videos to show how Retrieval-augmented generation (RAG) system could work for Cypress test writing.
🤖🎉













Hello Gleb. I have been a religious follower of your Cypress posts. However, I was wondering if you also have any plans of sharing posts related to playwright. The reason I ask - I see a shift from Cypress to Playwright and I am a heavy user of Playwright now. Do share your thoughts on it. Thanks