From df0e45a5198b6ae3ff6d71e7d1cf3688d1d0cda0 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka Date: Tue, 10 Jun 2025 12:24:12 +0200 Subject: [PATCH 01/45] Tutorials for Vector Search and HFE --- src/manifest.json | 20 ++++ src/uc/ai_assistant.md | 94 +++++++++++++++ src/uc/personalized_recommendations.md | 160 +++++++++++++++++++++++++ 3 files changed, 274 insertions(+) create mode 100644 src/uc/ai_assistant.md create mode 100644 src/uc/personalized_recommendations.md diff --git a/src/manifest.json b/src/manifest.json index 3c37234..fe0cccc 100644 --- a/src/manifest.json +++ b/src/manifest.json @@ -34,6 +34,26 @@ "initialIsOpen": true, "path": "/uc/rag.md" } + }, + { + "type": "internal-link", + "id": "ai_assistant", + "label": "Creating an AI Assistant", + "summary": "An assistant that temporarily retains conversations and understands by their meaning.", + "args": { + "initialIsOpen": true, + "path": "/uc/ai_assistant.md" + } + }, + { + "type": "internal-link", + "id": "redis_use_cases_rag", + "label": "Building personalized recommendations", + "summary": "Not just keyword matching, but semantic understanding.", + "args": { + "initialIsOpen": true, + "path": "/uc/personalized_recommendations.md" + } } ] }, diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md new file mode 100644 index 0000000..384369f --- /dev/null +++ b/src/uc/ai_assistant.md @@ -0,0 +1,94 @@ +Imagine you are building a smart AI assistant that: + - Remembers chats, but only temporarily + - Thinks by meaning, not by matching exact words + - Cleans up after itself, with no manual scripts + +Redis 8 has two powerful capabilities to make this happen: + - Field-level expiration: Let individual chat messages expire on their own + - Vector similarity search: Find past messages based on meaning, not keywords + +Let’s dive in. + +### Short-Term Memory with Field-Level Expiry +Each chat session is stored as a Redis Hash. +Each message is a field in the hash. +Redis 8’s new HSETEX command allows you to assign a TTL to each field, perfect for building ephemeral, session-level memory. + +```redis:[run_confirmation=true] Upload Session Data +// session:42 is the session ID +// msg: ensures uniqueness and traceability +HSETEX session:42 msg:1717935301 120 "Hi Chatbot!" +HSETEX session:42 msg:1717935361 180 "What can you do?" +HSETEX session:42 msg:1717935440 90 "Can you remind me about my tasks?" +HSETEX session:42 msg:1717935720 30 "What's the news today?" +``` + +Each field automatically expires after its TTL (in seconds). +No need for cron jobs or background workers. +What you get: + - Clean memory + - Zero manual cleanup + - Session-scoped retention, just like short-term memory in humans + + +Try it: After a few minutes, run `HGETALL session:42` and see what's left. + +### Vector Search for Semantic Recall +Now, your assistant needs to “recall” semantically related messages, not just match by words. +To do that, you’ll: + - Convert messages to vector embeddings + - Store them in Redis + - Use Vector Search with FT.SEARCH for semantic retrieval + +```redis:[run_confirmation=true] Create a Vector Index +FT.CREATE idx:memory ON HASH PREFIX 1 memory: SCHEMA + message TEXT + embedding VECTOR FLAT // FLAT = exact vector search + 6 + TYPE FLOAT32 + DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. + DISTANCE_METRIC COSINE // COSINE = measures semantic closeness +``` + +Now, let’s add entries for your chatbots: + +```redis:[run_confirmation=true] Add entries for the chatbot +// Embeddings are stored as binary FLOAT32 vectors - this is a compact format required by Redis Vector Serch indexes +HSET memory:1 message "Book a dentist appointment" embedding "\x00\x00\x80?\x00\x00\x00@\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" +HSET memory:2 message "Remind me to water plants" embedding "\x00\x00\x80@\x00\x00\x80@\x00\x00\x80@\x00\x00\x80?\x00\x00\x80?\x00\x00@@" +HSET memory:3 message "What’s the weather like?" embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x00@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80?" +HSET memory:4 message "Cancel my gym session" embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" +HSET memory:5 message "Start a new shopping list" embedding "\x00\x00\x00@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80@\x00\x00\x80?\x00\x00@@" +``` + +Now your messages are vectorized and ready for search. + +### Let Chatbot Think – Semantic Search with Vectors +When a user sends a new message, convert it to an embedding and run a KNN search: + +```redis:[run_confirmation=true] Search For Similar Messages +// Returns the top 3 semantically similar messages, even if no words match directly. +FT.SEARCH idx:memory "*=>[KNN 3 @embedding $vec AS score]" + PARAMS 2 vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" + SORTBY score + DIALECT 2 +``` + +Now your assistant “remembers” things it’s heard before - by meaning. + +### Real-Time Session Cleanup – Redis Handles It +Want to check what's still in memory? + +```redis:[run_confirmation=false] Check Sessions +HGETALL session:42 +``` + +Only the unexpired fields remain. Redis does the cleanup invisibly in the background. +Your assistant has a clean, focused mind at all times. + +### Next Steps +Now that your assistant has memory and meaning, you can: + - Tie session messages to store embeddings for per-session recall + - Use RAG (Retrieval-Augmented Generation) by combining Redis Vector Search with LLMs + - Add per-user memory: prefix session keys with a user ID (user:42:session:...) + - Introduce a fallback to persistent storage for long-term memory using Redis Flex \ No newline at end of file diff --git a/src/uc/personalized_recommendations.md b/src/uc/personalized_recommendations.md new file mode 100644 index 0000000..020c4ce --- /dev/null +++ b/src/uc/personalized_recommendations.md @@ -0,0 +1,160 @@ +Imagine you’re building a movie recommendation app, and your users expect intuitive, meaningful results, not just keyword matches, but intelligent understanding. + +Not just keyword matching, but semantic understanding — powered by vector embeddings. These are numerical representations of text that capture meaning, allowing you to search by intent rather than exact words. + +### Store Movie Documents with Vector Embeddings + +Let’s import a dataset containing plot summaries, each paired with an embedding vector. Each movie JSON document contains metadata like title, genre, year, and a vector embedding of the plot. + +```redis:[run_confirmation=true] Upload Movies +JSON.SET movie:001 $ '{"title":"Toy Story","genres":["Animation","Comedy","Family"],"plot":"Toys come to life when humans arent around.","year":1995,"embedding":[0.22,0.04,0.33,0.12,-0.02,0.17,0.09,0.01]}' +JSON.SET movie:002 $ '{"title":"Inside Out","genres":["Animation","Comedy","Drama"],"plot":"Emotions guide a young girl through change.","year":2015,"embedding":[0.20,0.03,0.31,0.11,-0.03,0.16,0.08,0.02]}' +JSON.SET movie:003 $ '{"title":"Whiplash","genres":["Drama","Music"],"plot":"A young drummer is pushed to greatness.","year":2014,"embedding":[0.14,0.01,0.22,0.08,-0.07,0.10,0.04,0.00]}' +JSON.SET movie:004 $ '{"title":"La La Land","genres":["Drama","Music","Romance"],"plot":"A jazz musician falls in love in LA.","year":2016,"embedding":[0.15,0.03,0.23,0.09,-0.08,0.14,0.06,0.01]}' +JSON.SET movie:005 $ '{"title":"The Matrix","genres":["Action","Sci-Fi"],"plot":"A hacker discovers reality is a simulation.","year":1999,"embedding":[0.12,-0.03,0.25,0.04,-0.10,0.09,0.05,-0.02]}' +JSON.SET movie:006 $ '{"title":"Inception","genres":["Action","Adventure","Sci-Fi"],"plot":"A thief steals information through dreams.","year":2010,"embedding":[0.14,-0.01,0.27,0.06,-0.09,0.10,0.04,-0.03]}' +JSON.SET movie:007 $ '{"title":"Tenet","genres":["Action","Sci-Fi","Thriller"],"plot":"Time-inversion to prevent World War III.","year":2020,"embedding":[0.13,-0.06,0.29,0.05,-0.11,0.12,0.06,-0.01]}' +JSON.SET movie:008 $ '{"title":"Finding Nemo","genres":["Animation","Adventure","Family"],"plot":"A clownfish searches for his son.","year":2003,"embedding":[0.18,0.02,0.30,0.10,-0.05,0.15,0.07,0.01]}' +JSON.SET movie:009 $ '{"title":"Coco","genres":["Animation","Family","Music"],"plot":"A boy enters the Land of the Dead.","year":2017,"embedding":[0.21,0.04,0.34,0.13,-0.02,0.19,0.10,0.02]}' +JSON.SET movie:010 $ '{"title":"Soul","genres":["Animation","Adventure","Comedy"],"plot":"A jazz musician explores the afterlife.","year":2020,"embedding":[0.16,0.02,0.28,0.10,-0.06,0.13,0.07,0.00]}' +JSON.SET movie:011 $ '{"title":"The Dark Knight","genres":["Action","Crime","Drama"],"plot":"Batman fights the Joker.","year":2008,"embedding":[0.12,-0.03,0.25,0.04,-0.09,0.10,0.05,-0.02]}' +JSON.SET movie:012 $ '{"title":"Frozen","genres":["Animation","Adventure","Comedy"],"plot":"A princess sets off to find her sister.","year":2013,"embedding":[0.22,0.04,0.33,0.12,-0.03,0.18,0.10,0.02]}' +JSON.SET movie:013 $ '{"title":"The Lion King","genres":["Animation","Adventure","Drama"],"plot":"A lion prince flees and returns.","year":1994,"embedding":[0.19,0.02,0.32,0.10,-0.04,0.18,0.09,0.03]}' +JSON.SET movie:014 $ '{"title":"Shrek","genres":["Animation","Adventure","Comedy"],"plot":"An ogre rescues a princess.","year":2001,"embedding":[0.21,0.03,0.32,0.11,-0.04,0.17,0.09,0.01]}' +JSON.SET movie:015 $ '{"title":"The Social Network","genres":["Biography","Drama"],"plot":"The rise of Facebook and its creator.","year":2010,"embedding":[0.10,-0.01,0.21,0.05,-0.07,0.06,0.03,-0.02]}' +JSON.SET movie:016 $ '{"title":"Guardians of the Galaxy","genres":["Action","Adventure","Sci-Fi"],"plot":"A group of intergalactic criminals must save the universe.","year":2014,"embedding":[0.13,0.00,0.28,0.07,-0.08,0.11,0.05,-0.01]}' +JSON.SET movie:017 $ '{"title":"Moana","genres":["Animation","Adventure","Family"],"plot":"A young girl sets sail to save her island.","year":2016,"embedding":[0.20,0.03,0.33,0.12,-0.03,0.17,0.09,0.02]}' +JSON.SET movie:018 $ '{"title":"Whale Rider","genres":["Drama","Family"],"plot":"A girl fights tradition to become chief.","year":2002,"embedding":[0.15,0.01,0.25,0.09,-0.05,0.12,0.06,0.00]}' +JSON.SET movie:019 $ '{"title":"Rocketman","genres":["Biography","Drama","Music"],"plot":"The story of Elton Johns breakthrough years.","year":2019,"embedding":[0.14,0.01,0.22,0.07,-0.06,0.11,0.05,0.01]}' +JSON.SET movie:020 $ '{"title":"Amadeus","genres":["Biography","Drama","Music"],"plot":"The rivalry between Mozart and Salieri.","year":1984,"embedding":[0.13,0.00,0.20,0.06,-0.07,0.10,0.04,0.00]}' +JSON.SET movie:021 $ '{"title":"The Sound of Music","genres":["Biography","Drama","Music"],"plot":"A governess brings music to a family.","year":1965,"embedding":[0.14,0.02,0.21,0.07,-0.06,0.11,0.05,0.01]}' +JSON.SET movie:022 $ '{"title":"Les Miserables","genres":["Drama","Music","Romance"],"plot":"The struggles of ex-convict Jean Valjean.","year":2012,"embedding":[0.13,0.01,0.23,0.08,-0.07,0.12,0.05,0.01]}' +JSON.SET movie:023 $ '{"title":"The Greatest Showman","genres":["Biography","Drama","Music"],"plot":"The story of P.T. Barnum and his circus.","year":2017,"embedding":[0.15,0.03,0.25,0.09,-0.05,0.13,0.06,0.02]}' +JSON.SET movie:024 $ '{"title":"A Star Is Born","genres":["Drama","Music","Romance"],"plot":"A musician helps a young singer find fame.","year":2018,"embedding":[0.14,0.02,0.24,0.08,-0.06,0.12,0.05,0.01]}' +JSON.SET movie:025 $ '{"title":"Mad Max: Fury Road","genres":["Action","Adventure","Sci-Fi"],"plot":"In a post-apocalyptic wasteland, Max helps rebels escape.","year":2015,"embedding":[0.11,-0.02,0.26,0.05,-0.10,0.08,0.05,-0.02]}' +JSON.SET movie:026 $ '{"title":"Blade Runner 2049","genres":["Sci-Fi","Thriller"],"plot":"A new blade runner uncovers secrets.","year":2017,"embedding":[0.12,-0.03,0.27,0.06,-0.09,0.09,0.06,-0.01]}' +JSON.SET movie:027 $ '{"title":"Arrival","genres":["Drama","Sci-Fi","Thriller"],"plot":"A linguist communicates with aliens.","year":2016,"embedding":[0.13,-0.01,0.28,0.07,-0.08,0.11,0.05,-0.01]}' +JSON.SET movie:028 $ '{"title":"Interstellar","genres":["Adventure","Drama","Sci-Fi"],"plot":"Explorers travel through a wormhole in space.","year":2014,"embedding":[0.14,-0.02,0.29,0.08,-0.09,0.12,0.06,-0.02]}' +JSON.SET movie:029 $ '{"title":"E.T. the Extra-Terrestrial","genres":["Family","Sci-Fi"],"plot":"A boy befriends an alien.","year":1982,"embedding":[0.17,0.01,0.31,0.10,-0.06,0.15,0.07,0.01]}' +JSON.SET movie:030 $ '{"title":"The Avengers","genres":["Action","Adventure","Sci-Fi"],"plot":"Superheroes team up to save the world.","year":2012,"embedding":[0.13,0.00,0.27,0.07,-0.08,0.11,0.06,-0.01]}' +JSON.SET movie:031 $ '{"title":"Guardians of the Galaxy Vol. 2","genres":["Action","Adventure","Comedy"],"plot":"The Guardians fight to protect the galaxy.","year":2017,"embedding":[0.15,0.01,0.28,0.09,-0.07,0.13,0.07,0.01]}' +JSON.SET movie:032 $ '{"title":"Up","genres":["Animation","Adventure","Comedy"],"plot":"An old man goes on an adventure in his flying house.","year":2009,"embedding":[0.21,0.04,0.32,0.11,-0.04,0.16,0.09,0.02]}' +JSON.SET movie:033 $ '{"title":"Zootopia","genres":["Animation","Adventure","Comedy"],"plot":"A bunny cop solves a mystery in a city of animals.","year":2016,"embedding":[0.20,0.03,0.31,0.10,-0.05,0.15,0.08,0.01]}' +JSON.SET movie:034 $ '{"title":"Big Hero 6","genres":["Animation","Action","Comedy"],"plot":"A robotics prodigy teams with friends to fight crime.","year":2014,"embedding":[0.19,0.02,0.30,0.09,-0.05,0.14,0.08,0.01]}' +JSON.SET movie:035 $ '{"title":"The Prestige","genres":["Drama","Mystery","Sci-Fi"],"plot":"Two magicians engage in a deadly rivalry.","year":2006,"embedding":[0.12,-0.02,0.24,0.06,-0.08,0.10,0.05,-0.01]}' +JSON.SET movie:036 $ '{"title":"Dunkirk","genres":["Action","Drama","History"],"plot":"Allied soldiers are evacuated during WWII.","year":2017,"embedding":[0.10,-0.03,0.22,0.05,-0.09,0.07,0.04,-0.02]}' +JSON.SET movie:037 $ '{"title":"Jumanji: Welcome to the Jungle","genres":["Action","Adventure","Comedy"],"plot":"Teens trapped in a video game jungle.","year":2017,"embedding":[0.16,0.01,0.27,0.08,-0.06,0.12,0.06,0.01]}' +JSON.SET movie:038 $ '{"title":"Cinderella","genres":["Animation","Family","Fantasy"],"plot":"A young girl overcomes her cruel stepmother.","year":1950,"embedding":[0.19,0.03,0.31,0.11,-0.04,0.16,0.08,0.02]}' +JSON.SET movie:039 $ '{"title":"Mulan","genres":["Animation","Adventure","Drama"],"plot":"A young woman disguises as a soldier.","year":1998,"embedding":[0.20,0.03,0.32,0.11,-0.04,0.17,0.09,0.02]}' +JSON.SET movie:040 $ '{"title":"Beauty and the Beast","genres":["Animation","Family","Fantasy"],"plot":"A young woman falls in love with a beast.","year":1991,"embedding":[0.18,0.02,0.30,0.10,-0.05,0.15,0.08,0.01]}' +JSON.SET movie:041 $ '{"title":"The Godfather","genres":["Crime","Drama"],"plot":"The aging patriarch of an organized crime dynasty transfers control to his son.","year":1972,"embedding":[0.11,-0.04,0.24,0.06,-0.10,0.07,0.05,-0.03]}' +JSON.SET movie:042 $ '{"title":"Pulp Fiction","genres":["Crime","Drama"],"plot":"The lives of two mob hitmen, a boxer, and others intertwine.","year":1994,"embedding":[0.12,-0.03,0.23,0.07,-0.09,0.09,0.04,-0.01]}' +JSON.SET movie:043 $ '{"title":"Forrest Gump","genres":["Drama","Romance"],"plot":"The presidencies of Kennedy and Johnson through the eyes of Forrest.","year":1994,"embedding":[0.14,0.01,0.26,0.08,-0.07,0.11,0.06,0.01]}' +JSON.SET movie:044 $ '{"title":"Gladiator","genres":["Action","Drama"],"plot":"A former Roman General seeks revenge.","year":2000,"embedding":[0.13,0.00,0.25,0.07,-0.08,0.10,0.05,0.00]}' +JSON.SET movie:045 $ '{"title":"Titanic","genres":["Drama","Romance"],"plot":"A seventeen-year-old aristocrat falls in love with a kind but poor artist.","year":1997,"embedding":[0.15,0.02,0.28,0.09,-0.06,0.13,0.06,0.01]}' +JSON.SET movie:046 $ '{"title":"Jurassic Park","genres":["Adventure","Sci-Fi"],"plot":"Scientists clone dinosaurs for a theme park.","year":1993,"embedding":[0.14,-0.01,0.26,0.08,-0.07,0.11,0.06,0.00]}' +JSON.SET movie:047 $ '{"title":"The Shawshank Redemption","genres":["Drama"],"plot":"Two imprisoned men bond over a number of years.","year":1994,"embedding":[0.15,0.00,0.27,0.09,-0.06,0.12,0.07,0.01]}' +JSON.SET movie:048 $ '{"title":"Fight Club","genres":["Drama"],"plot":"An insomniac and a soap maker form an underground fight club.","year":1999,"embedding":[0.13,-0.02,0.24,0.07,-0.08,0.10,0.05,-0.01]}' +JSON.SET movie:049 $ '{"title":"The Silence of the Lambs","genres":["Crime","Drama","Thriller"],"plot":"A young FBI cadet seeks help from an imprisoned cannibal.","year":1991,"embedding":[0.11,-0.03,0.22,0.06,-0.09,0.08,0.04,-0.02]}' +JSON.SET movie:050 $ '{"title":"The Departed","genres":["Crime","Drama","Thriller"],"plot":"An undercover cop and a mole in the police attempt to identify each other.","year":2006,"embedding":[0.12,-0.02,0.23,0.07,-0.08,0.09,0.05,-0.01]}' +JSON.SET movie:51 $ '{"title":"Saturday Night Fever","year":1977,"genres":["Drama","Music"],"plot":"A young man finds escape from his mundane life through disco dancing.","embedding":[0.154,-0.050,0.300,0.150,-0.150,0.154,0.034,-0.118]}' +JSON.SET movie:52 $ '{"title":"The Rose","year":1979,"genres":["Music","Drama"],"plot":"A rock star struggles with fame, addiction, and love.","embedding":[0.144,-0.055,0.295,0.145,-0.160,0.150,0.030,-0.120]}' +JSON.SET movie:53 $ '{"title":"Cabaret","year":1972,"genres":["Drama","Music"],"plot":"A performer and a writer navigate love and politics in pre-WWII Berlin.","embedding":[0.151,-0.052,0.297,0.148,-0.152,0.150,0.031,-0.121]}' +JSON.SET movie:54 $ '{"title":"Tommy","year":1975,"genres":["Drama","Music","Fantasy"],"plot":"A deaf and blind boy becomes a pinball champion and religious figure.","embedding":[0.149,-0.051,0.301,0.149,-0.151,0.153,0.033,-0.119]}' +JSON.SET movie:55 $ '{"title":"All That Jazz","year":1979,"genres":["Drama","Music"],"plot":"A choreographer reflects on his life and art while facing death.","embedding":[0.153,-0.049,0.299,0.151,-0.149,0.152,0.032,-0.117]}' +``` + +### Create a Vector-Enabled Search Index +Redis stores movie data as JSON documents with text fields (title, genres, plot) and vector embeddings. Creating an index lets you quickly filter documents and run fast approximate nearest neighbor (ANN) searches on embeddings. + +```redis:[run_confirmation=true] Create a Vector Index +FT.CREATE idx:movies ON JSON PREFIX 1 "movie:" SCHEMA + $.title AS title TEXT + $.genres[*] AS genres TAG + $.plot AS plot TEXT + $.year AS year NUMERIC + $.embedding AS embedding VECTOR FLAT + 6 + TYPE FLOAT32 + DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. + DISTANCE_METRIC COSINE // COSINE = measures semantic closeness + ``` + +This sets the stage for combined textual and semantic search. + +### Semantic Search by Query Embedding +When users search: “I want a fun animated movie about toys and friendship.”, you can generate embeddings using models like text-embedding-3-small from OpenAI, or sentence-transformers from Hugging Face. These models turn text into numerical vectors you can store and query with Redis. + +```redis:[run_confirmation=false] Search Per Plot +FT.SEARCH idx:movies "*=>[KNN 3 @embedding $vec AS score]" + PARAMS 2 vec "\x9a\x99\x19\x3f\xcd\xcc\xcc\x3d\x9a\x99\x4c\x3f\x9a\x99\x33\x3e\x9a\x99\x33\x3f\xcd\xcc\x66\x3e\xcd\xcc\xcc\x3d\xcd\xcc\x4c\x3e" + SORTBY score + RETURN 3 title plot score + DIALECT 2 +``` + +Redis returns top movies with embeddings close to the query vector - Toy Story ranks first, even if keywords don’t exactly match. + +### Adding Filters for Hybrid Search + +Now, let’s find music movies: +“A feel-good film about music and students.” + +You can combine a genre filter with vector similarity: + +```redis:[run_confirmation=false] Search Per Genre +FT.SEARCH idx:movies "@genres:{Music} =>[KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" + SORTBY score + RETURN 3 title genres score + DIALECT 2 +``` + +This hybrid query uses Redis’s tagging system plus vector search, improving relevance. + +### Using Embeddings of Existing Movies for Recommendations + +Let’s say you love Inception and want similar movies. Let’s retrieve the Inception plot embedding and use it as the query vector: +```redis:[run_confirmation=false] Get the Embedding From the Movie Document +JSON.GET movie:006 $.embedding +``` +Now let’s run vector similarity search using that embedding as a binary blob (FLOAT32-packed): + +```redis:[run_confirmation=false] Search for Similar Movies +FT.SEARCH idx:movies "*=>[KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\xCD\xCC\x56\x3E\x9A\x99\xF3\xBC\xCD\xCC\x00\x3F\x66\x66\x34\x3E\xC6\xF5\x1B\xBE\x9A\x99\x4D\x3E\x9A\x99\x99\x3D\x9A\x99\xB5\xBD" + SORTBY score + RETURN 2 title score + DIALECT 2 +``` + +Redis finds movies like Arrival and The Departed, showcasing content-based recommendation with semantic similarity. + +### Combining Metadata Filters with Vector Search +Let’s find classic musical rebellion films from the 90s: +```redis:[run_confirmation=false] Search for Classical Musical Rebellion Films +FT.SEARCH idx:movies "(@genres:{Music} @year:[1970 1979]) =>[KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" + SORTBY score + RETURN 4 title year genres score + DIALECT 2 +``` +This shows how Redis vector search works seamlessly with numeric and tag filters. + +### Personalizing Recommendations +Say we like Animated and Sci-Fi movies. You can personalize results by filtering the vector search: +```redis:[run_confirmation=false] Search Per Genres +FT.SEARCH idx:movies '@genres:{"Animated"|"Sci-Fi"} =>[KNN 5 @embedding $vec AS score]' + PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" + SORTBY score + RETURN 3 title genres score + DIALECT 2 +``` + +This makes Redis recommendations responsive to evolving user preferences without retraining embeddings. + +### Next Steps + - Now that you’ve seen the basics, here are a few ideas to extend this: + - Build personalized watchlists based on user preferences and past behavior. + - Power chatbots that provide context-aware movie suggestions. + - Blend keyword and vector search for a richer discovery experience. From fbaa0b1b7556c6009961f636230815639d66decc Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka Date: Tue, 10 Jun 2025 16:56:35 +0200 Subject: [PATCH 02/45] Changing the AI tutorial --- src/uc/ai_assistant.md | 185 ++++++++++++++++++++++++++--------------- 1 file changed, 120 insertions(+), 65 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index 384369f..0562b5f 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -1,94 +1,149 @@ -Imagine you are building a smart AI assistant that: - - Remembers chats, but only temporarily - - Thinks by meaning, not by matching exact words - - Cleans up after itself, with no manual scripts +Build a production-ready AI assistant that remembers conversations semantically and auto-cleans using Redis 8’s new features. -Redis 8 has two powerful capabilities to make this happen: - - Field-level expiration: Let individual chat messages expire on their own - - Vector similarity search: Find past messages based on meaning, not keywords +This smart AI assistant offers: + - **Ephemeral Memory**: Messages expire automatically at different intervals + - **Semantic Recall**: Search past chats by meaning, not just keywords + - **Zero Maintenance**: No cleanup scripts or cron jobs needed + - **Multi-User Support**: Memory isolated by user and session + - **Hybrid Search**: Combine text and vector search for better results -Let’s dive in. +**Note**: [Redis 8](https://hub.docker.com/_/redis/tags) is required for this tutorial Redis 8 since it introduces `HSETEX`, which lets you set TTL per hash field, ideal for ephemeral chat memory without complex expiration logic. -### Short-Term Memory with Field-Level Expiry -Each chat session is stored as a Redis Hash. -Each message is a field in the hash. -Redis 8’s new HSETEX command allows you to assign a TTL to each field, perfect for building ephemeral, session-level memory. +### Hierarchical Memory Structure +Store chat sessions as Redis hashes. Each message is a field with its own TTL. ```redis:[run_confirmation=true] Upload Session Data -// session:42 is the session ID -// msg: ensures uniqueness and traceability -HSETEX session:42 msg:1717935301 120 "Hi Chatbot!" -HSETEX session:42 msg:1717935361 180 "What can you do?" -HSETEX session:42 msg:1717935440 90 "Can you remind me about my tasks?" -HSETEX session:42 msg:1717935720 30 "What's the news today?" +// User-scoped sessions for isolation +// Pattern: user:{user_id}:session:{session_id} +HSETEX user:alice:session:morning msg:1717935301 3600 "Good morning! What's my schedule today?" +HSETEX user:alice:session:morning msg:1717935361 1800 "Remind me about the team meeting at 2 PM" +HSETEX user:alice:session:morning msg:1717935420 900 "What's the weather forecast?" +HSETEX user:alice:session:morning msg:1717935480 300 "Thanks, that's all for now" + +// Different user, same session pattern +HSETEX user:bob:session:work msg:1717935500 7200 "I need to prepare for the client presentation" +HSETEX user:bob:session:work msg:1717935560 3600 "What are the key points I should cover?" + ``` -Each field automatically expires after its TTL (in seconds). -No need for cron jobs or background workers. -What you get: - - Clean memory - - Zero manual cleanup - - Session-scoped retention, just like short-term memory in humans +### Memory Tiers for Different Lifetimes +Control how long messages last depending on their importance. + +```redis:[run_confirmation=true] Memory Tiers Strategy +// Short-term (5 minutes) - Immediate context +HSETEX user:alice:session:current msg:1717935301 300 "Current conversation context" + +// Medium-term (30 minutes) - Session memory +HSETEX user:alice:session:current msg:1717935302 1800 "Important session details" + +// Long-term (2 hours) - Cross-session context +HSETEX user:alice:session:current msg:1717935303 7200 "Key user preferences and facts" +``` +### Check Current Session Memory +No manual cleanup needed; expired messages vanish automatically. -Try it: After a few minutes, run `HGETALL session:42` and see what's left. +```redis:[run_confirmation=true] Monitor Session State Over Time +// After a few minutes, run this command to see what's left. +HGETALL user:alice:session:morning +``` -### Vector Search for Semantic Recall -Now, your assistant needs to “recall” semantically related messages, not just match by words. -To do that, you’ll: - - Convert messages to vector embeddings - - Store them in Redis - - Use Vector Search with FT.SEARCH for semantic retrieval +### Vector Search Setup for Semantic Recall +Create an index to store messages as vectors for semantic search. ```redis:[run_confirmation=true] Create a Vector Index -FT.CREATE idx:memory ON HASH PREFIX 1 memory: SCHEMA - message TEXT - embedding VECTOR FLAT // FLAT = exact vector search - 6 - TYPE FLOAT32 - DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. - DISTANCE_METRIC COSINE // COSINE = measures semantic closeness +FT.CREATE idx:ai_memory + ON HASH + PREFIX 1 memory: + SCHEMA + user_id TAG SORTABLE + session_id TAG SORTABLE + message TEXT WEIGHT 2.0 PHONETIC dm:en + context TEXT WEIGHT 1.0 + timestamp NUMERIC SORTABLE + embedding VECTOR HNSW 6 + TYPE FLOAT32 + DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. + DISTANCE_METRIC COSINE // COSINE = measures semantic closeness + INITIAL_CAP 10000 + M 16 + EF_CONSTRUCTION 200 ``` -Now, let’s add entries for your chatbots: +Add sample vectorized messages (embedding dims are demo-sized): ```redis:[run_confirmation=true] Add entries for the chatbot -// Embeddings are stored as binary FLOAT32 vectors - this is a compact format required by Redis Vector Serch indexes -HSET memory:1 message "Book a dentist appointment" embedding "\x00\x00\x80?\x00\x00\x00@\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" -HSET memory:2 message "Remind me to water plants" embedding "\x00\x00\x80@\x00\x00\x80@\x00\x00\x80@\x00\x00\x80?\x00\x00\x80?\x00\x00@@" -HSET memory:3 message "What’s the weather like?" embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x00@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80?" -HSET memory:4 message "Cancel my gym session" embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" -HSET memory:5 message "Start a new shopping list" embedding "\x00\x00\x00@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80@\x00\x00\x80?\x00\x00@@" +HSET memory:alice:1 user_id "alice" session_id "morning" message "I have a dentist appointment at 3 PM today" context "healthcare scheduling appointment" timestamp 1717935301 embedding "\x00\x00\x80?\x00\x00\x00@\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" +HSET memory:alice:2 user_id "alice" session_id "morning" message "Remind me to water the plants in my office" context "task reminder plants office" timestamp 1717935361 embedding "\x00\x00\x80@\x00\x00\x80@\x00\x00\x80@\x00\x00\x80?\x00\x00\x80?\x00\x00@@" +HSET memory:alice:3 user_id "alice" session_id "work" message "Schedule a meeting with the engineering team" context "work scheduling meeting team" timestamp 1717935420 embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x00@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80?" +HSET memory:bob:1 user_id "bob" session_id "work" message "I need to review the quarterly sales report" context "business analysis quarterly report" timestamp 1717935480 embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" ``` -Now your messages are vectorized and ready for search. - ### Let Chatbot Think – Semantic Search with Vectors -When a user sends a new message, convert it to an embedding and run a KNN search: - -```redis:[run_confirmation=true] Search For Similar Messages -// Returns the top 3 semantically similar messages, even if no words match directly. -FT.SEARCH idx:memory "*=>[KNN 3 @embedding $vec AS score]" - PARAMS 2 vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - SORTBY score - DIALECT 2 +When a user says something new, find all related past conversations across your entire system based on semantic meaning. + +```redis:[run_confirmation=false] Find Top 5 Related Messages By Meaning +FT.SEARCH idx:ai_memory + "*=>[KNN 5 @embedding $query_vec AS vector_score]" + PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" + RETURN 6 user_id message context vector_score timestamp + SORTBY vector_score ASC + DIALECT 2 ``` Now your assistant “remembers” things it’s heard before - by meaning. -### Real-Time Session Cleanup – Redis Handles It -Want to check what's still in memory? +### User-Scoped Semantic Search +Your AI should only recall memories from the specific user it's talking to, not leak information between users. + +```redis:[run_confirmation=false] Find Similar Memories For Specific User Only +FT.SEARCH idx:ai_memory + "(@user_id:{alice}) => [KNN 3 @embedding $query_vec AS vector_score]" + PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" + RETURN 6 user_id message context vector_score session_id + SORTBY vector_score ASC + DIALECT 2 +``` + +### Time-Bounded Semantic Search +When users ask about "recent" things, limit your search to a specific time window while still using semantic matching. + +```redis:[run_confirmation=false] Find recent similar memories (last 24 hours) +FT.SEARCH idx:ai_memory + "(@timestamp:[1717849200 +inf]) => [KNN 3 @embedding $query_vec AS vector_score]" + PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" + RETURN 6 message timestamp vector_score + SORTBY timestamp DESC + DIALECT 2 +``` + +### Session-Specific Recall +When users refer to something from "earlier in our conversation," search only within the current session context. -```redis:[run_confirmation=false] Check Sessions -HGETALL session:42 ``` +FT.SEARCH idx:ai_memory + "(@user_id:{alice} @session_id:{morning}) => [KNN 10 @embedding $query_vec AS vector_score]" + PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" + RETURN 6 message context vector_score timestamp + SORTBY vector_score ASC + DIALECT 2 +``` + +### Want to check what's still in memory? + +Only unexpired fields remain. -Only the unexpired fields remain. Redis does the cleanup invisibly in the background. -Your assistant has a clean, focused mind at all times. +```redis:[run_confirmation=false] Check Sessions +HGETALL memory:alice:1 +HGETALL memory:alice:2 +HGETALL memory:alice:3 +HGETALL memory:bob:1 +HGETALL memory:bob:2 +``` ### Next Steps Now that your assistant has memory and meaning, you can: - - Tie session messages to store embeddings for per-session recall - - Use RAG (Retrieval-Augmented Generation) by combining Redis Vector Search with LLMs - - Add per-user memory: prefix session keys with a user ID (user:42:session:...) - - Introduce a fallback to persistent storage for long-term memory using Redis Flex \ No newline at end of file + - Combine Redis Vector Search with LLMs (RAG) + - Use OpenAI or sentence-transformers for embeddings + - Add fallback persistent storage with Redis Flex + - Manage users with ACLs, quotas, and keyspace notifications \ No newline at end of file From 1da215831c1cd8fa6e478dab8a85820c86ae5df4 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka Date: Tue, 10 Jun 2025 19:10:24 +0200 Subject: [PATCH 03/45] Updating the tutorial for AI chatbot --- src/uc/ai_assistant.md | 69 ++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 37 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index 0562b5f..740cac3 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -15,15 +15,14 @@ Store chat sessions as Redis hashes. Each message is a field with its own TTL. ```redis:[run_confirmation=true] Upload Session Data // User-scoped sessions for isolation // Pattern: user:{user_id}:session:{session_id} -HSETEX user:alice:session:morning msg:1717935301 3600 "Good morning! What's my schedule today?" -HSETEX user:alice:session:morning msg:1717935361 1800 "Remind me about the team meeting at 2 PM" -HSETEX user:alice:session:morning msg:1717935420 900 "What's the weather forecast?" -HSETEX user:alice:session:morning msg:1717935480 300 "Thanks, that's all for now" +HSETEX user:alice:session:morning EX 3600 FIELDS 1 msg:1717935301 "Good morning! What's my schedule today?" +HSETEX user:alice:session:morning EX 1800 FIELDS 1 msg:1717935361 "Remind me about the team meeting at 2 PM" +HSETEX user:alice:session:morning EX 900 FIELDS 1 msg:1717935420 "What's the weather forecast?" +HSETEX user:alice:session:morning EX 300 FIELDS 1 msg:1717935480 "Thanks, that's all for now" // Different user, same session pattern -HSETEX user:bob:session:work msg:1717935500 7200 "I need to prepare for the client presentation" -HSETEX user:bob:session:work msg:1717935560 3600 "What are the key points I should cover?" - +HSETEX user:bob:session:work EX 7200 FIELDS 1 msg:1717935500 "I need to prepare for the client presentation" +HSETEX user:bob:session:work EX 3600 FIELDS 1 msg:1717935560 "What are the key points I should cover?" ``` ### Memory Tiers for Different Lifetimes @@ -31,13 +30,13 @@ Control how long messages last depending on their importance. ```redis:[run_confirmation=true] Memory Tiers Strategy // Short-term (5 minutes) - Immediate context -HSETEX user:alice:session:current msg:1717935301 300 "Current conversation context" +HSETEX user:alice:session:current EX 300 FIELDS 1 msg:1717935301 "Current conversation context" // Medium-term (30 minutes) - Session memory -HSETEX user:alice:session:current msg:1717935302 1800 "Important session details" +HSETEX user:alice:session:current EX 1800 FIELDS 1 msg:1717935302 "Important session details" // Long-term (2 hours) - Cross-session context -HSETEX user:alice:session:current msg:1717935303 7200 "Key user preferences and facts" +HSETEX user:alice:session:current EX 7200 FIELDS 1 msg:1717935303 "Key user preferences and facts" ``` ### Check Current Session Memory @@ -52,43 +51,40 @@ HGETALL user:alice:session:morning Create an index to store messages as vectors for semantic search. ```redis:[run_confirmation=true] Create a Vector Index -FT.CREATE idx:ai_memory - ON HASH - PREFIX 1 memory: - SCHEMA +FT.CREATE idx:ai_memory + ON HASH + PREFIX 1 memory: + SCHEMA user_id TAG SORTABLE - session_id TAG SORTABLE - message TEXT WEIGHT 2.0 PHONETIC dm:en - context TEXT WEIGHT 1.0 + session_id TAG SORTABLE + message TEXT + context TEXT timestamp NUMERIC SORTABLE - embedding VECTOR HNSW 6 - TYPE FLOAT32 - DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. + embedding VECTOR HNSW 6 + TYPE FLOAT32 + DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 DISTANCE_METRIC COSINE // COSINE = measures semantic closeness - INITIAL_CAP 10000 - M 16 - EF_CONSTRUCTION 200 ``` Add sample vectorized messages (embedding dims are demo-sized): ```redis:[run_confirmation=true] Add entries for the chatbot -HSET memory:alice:1 user_id "alice" session_id "morning" message "I have a dentist appointment at 3 PM today" context "healthcare scheduling appointment" timestamp 1717935301 embedding "\x00\x00\x80?\x00\x00\x00@\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" -HSET memory:alice:2 user_id "alice" session_id "morning" message "Remind me to water the plants in my office" context "task reminder plants office" timestamp 1717935361 embedding "\x00\x00\x80@\x00\x00\x80@\x00\x00\x80@\x00\x00\x80?\x00\x00\x80?\x00\x00@@" -HSET memory:alice:3 user_id "alice" session_id "work" message "Schedule a meeting with the engineering team" context "work scheduling meeting team" timestamp 1717935420 embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x00@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80?" -HSET memory:bob:1 user_id "bob" session_id "work" message "I need to review the quarterly sales report" context "business analysis quarterly report" timestamp 1717935480 embedding "\x00\x00@@\x00\x00\x00@\x00\x00\x80?\x00\x00\x80@\x00\x00\x00@\x00\x00\x00@" +HSET memory:alice:1 user_id "alice" session_id "morning" message "I have a dentist appointment at 3 PM today" context "healthcare scheduling appointment" timestamp 1717935301 embedding "\x3f\x00\x00\x00\x40\x00\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00" +HSET memory:alice:2 user_id "alice" session_id "morning" message "Remind me to water the plants in my office" context "task reminder plants office" timestamp 1717935361 embedding "\x40\x00\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x3f\x00\x00\x00\x40\x00\x00\x00" +HSET memory:alice:3 user_id "alice" session_id "work" message "Schedule a meeting with the engineering team" context "work scheduling meeting team" timestamp 1717935420 embedding "\x40\x40\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x3f\x00\x00\x00" +HSET memory:bob:1 user_id "bob" session_id "work" message "I need to review the quarterly sales report" context "business analysis quarterly report" timestamp 1717935480 embedding "\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x00\x00\x00\x00" ``` ### Let Chatbot Think – Semantic Search with Vectors When a user says something new, find all related past conversations across your entire system based on semantic meaning. ```redis:[run_confirmation=false] Find Top 5 Related Messages By Meaning -FT.SEARCH idx:ai_memory - "*=>[KNN 5 @embedding $query_vec AS vector_score]" - PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - RETURN 6 user_id message context vector_score timestamp - SORTBY vector_score ASC - DIALECT 2 +FT.SEARCH idx:ai_memory + "*=>[KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" + RETURN 4 user_id message context timestamp + SORTBY score ASC + DIALECT 2 ``` Now your assistant “remembers” things it’s heard before - by meaning. @@ -100,7 +96,7 @@ Your AI should only recall memories from the specific user it's talking to, not FT.SEARCH idx:ai_memory "(@user_id:{alice}) => [KNN 3 @embedding $query_vec AS vector_score]" PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - RETURN 6 user_id message context vector_score session_id + RETURN 5 user_id message context vector_score session_id SORTBY vector_score ASC DIALECT 2 ``` @@ -124,8 +120,8 @@ When users refer to something from "earlier in our conversation," search only wi FT.SEARCH idx:ai_memory "(@user_id:{alice} @session_id:{morning}) => [KNN 10 @embedding $query_vec AS vector_score]" PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - RETURN 6 message context vector_score timestamp - SORTBY vector_score ASC + RETURN 4 message context vector_score timestamp + SORTBY vector_score DIALECT 2 ``` @@ -138,7 +134,6 @@ HGETALL memory:alice:1 HGETALL memory:alice:2 HGETALL memory:alice:3 HGETALL memory:bob:1 -HGETALL memory:bob:2 ``` ### Next Steps From 849c9517edd39930cdb811f9fb0370d8483dfae6 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka Date: Wed, 11 Jun 2025 12:29:45 +0200 Subject: [PATCH 04/45] Updating the AI assistand guide --- src/uc/ai_assistant.md | 218 +++++++++++++++++++++-------------------- 1 file changed, 112 insertions(+), 106 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index 740cac3..baee798 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -1,144 +1,150 @@ -Build a production-ready AI assistant that remembers conversations semantically and auto-cleans using Redis 8’s new features. - -This smart AI assistant offers: - - **Ephemeral Memory**: Messages expire automatically at different intervals - - **Semantic Recall**: Search past chats by meaning, not just keywords - - **Zero Maintenance**: No cleanup scripts or cron jobs needed - - **Multi-User Support**: Memory isolated by user and session - - **Hybrid Search**: Combine text and vector search for better results - -**Note**: [Redis 8](https://hub.docker.com/_/redis/tags) is required for this tutorial Redis 8 since it introduces `HSETEX`, which lets you set TTL per hash field, ideal for ephemeral chat memory without complex expiration logic. - -### Hierarchical Memory Structure -Store chat sessions as Redis hashes. Each message is a field with its own TTL. - -```redis:[run_confirmation=true] Upload Session Data -// User-scoped sessions for isolation -// Pattern: user:{user_id}:session:{session_id} -HSETEX user:alice:session:morning EX 3600 FIELDS 1 msg:1717935301 "Good morning! What's my schedule today?" -HSETEX user:alice:session:morning EX 1800 FIELDS 1 msg:1717935361 "Remind me about the team meeting at 2 PM" -HSETEX user:alice:session:morning EX 900 FIELDS 1 msg:1717935420 "What's the weather forecast?" -HSETEX user:alice:session:morning EX 300 FIELDS 1 msg:1717935480 "Thanks, that's all for now" - -// Different user, same session pattern -HSETEX user:bob:session:work EX 7200 FIELDS 1 msg:1717935500 "I need to prepare for the client presentation" -HSETEX user:bob:session:work EX 3600 FIELDS 1 msg:1717935560 "What are the key points I should cover?" +This intelligent AI assistant is designed to support real-world, multi-session use with Redis as its memory core. + +What you get: + - **Smart Memory**: Ephemeral context that expires automatically, long-term facts retained forever + - **Semantic Search**: Recall relevant info by meaning using vector search + - **Zero Maintenance**: Auto-expiring short-term memory without a need to track timestamps manually + - **Multi-User**: Isolated memory per user + - **Learning**: Assistant can "understand" each user better the more it's used + +**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes, which is ideal for managing short-term memory with precision. + +### Architecture Overview +| Layer | Description | +| ---------- | ---------- | +| `Working Memory`| `Short-term chat context (ephemeral)` | +| `Knowledge Base` | `Persistent facts, user preferences` | +| `Vector Search` | `Unified semantic recall across both layers` | + +### Working Memory (Ephemeral) +Stores recent user messages with TTL based on importance. Automatically expires to prevent bloat. +This uses `HSETEX`, a Redis 8 command that adds field-level expiration to hashes. It allows storing all temporary messages in a single key while managing TTLs per message, simplifying short-term memory management without needing multiple keys. + +```redis:[run_confirmation=true] Recent Conversations with TTL Based on Importance. +// Quick exchanges (5 min) +HSETEX user:alice:session:001 EX 300 FIELDS 1 msg:001 "What's the weather?" +// Session context (30 min) +HSETEX user:alice:session:001 EX 1800 FIELDS 1 msg:002 "I need a dentist appointment" +// Important decisions (2 hours) +HSETEX user:alice:session:001 EX 7200 FIELDS 1 msg:003 "Book it for Tuesday 2 PM" ``` -### Memory Tiers for Different Lifetimes -Control how long messages last depending on their importance. - -```redis:[run_confirmation=true] Memory Tiers Strategy -// Short-term (5 minutes) - Immediate context -HSETEX user:alice:session:current EX 300 FIELDS 1 msg:1717935301 "Current conversation context" - -// Medium-term (30 minutes) - Session memory -HSETEX user:alice:session:current EX 1800 FIELDS 1 msg:1717935302 "Important session details" - -// Long-term (2 hours) - Cross-session context -HSETEX user:alice:session:current EX 7200 FIELDS 1 msg:1717935303 "Key user preferences and facts" +### Knowledge Base (Persistent) +Long-term memory: stores important facts, user preferences, and context across sessions. These never expire. +`embedding` is a binary-encoded `FLOAT32[]` used for vector similarity that can be generated using sentence-transformers or similar libraries. Demo uses 8-dim vectors; production models typically use 128–1536 dimensions. + +```redis:[run_confirmation=true] Important User Information That Never Expires. +// User preferences - need vector fields for search +HSET user:alice:knowledge:pref:001 user_id "alice" memory_type "knowledge" content "prefers mornings before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" +HSET user:alice:knowledge:pref:002 user_id "alice" memory_type "knowledge" content "likes detailed explanations" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +// Personal facts +HSET user:alice:knowledge:personal:001 user_id "alice" memory_type "knowledge" content "allergic to shellfish" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +HSET user:alice:knowledge:personal:002 user_id "alice" memory_type "knowledge" content "golden retriever named Max" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" +// Work context +HSET user:alice:knowledge:work:001 user_id "alice" memory_type "knowledge" content "Senior PM at TechCorp" importance 8 timestamp 1717935000 embedding "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" +HSET user:alice:knowledge:work:002 user_id "alice" memory_type "knowledge" content "leading Project Apollo" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x80\x00\x00\x3f\x40\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" ``` -### Check Current Session Memory -No manual cleanup needed; expired messages vanish automatically. - -```redis:[run_confirmation=true] Monitor Session State Over Time -// After a few minutes, run this command to see what's left. -HGETALL user:alice:session:morning -``` - -### Vector Search Setup for Semantic Recall -Create an index to store messages as vectors for semantic search. +### Vector Search: Semantic Memory Recall +Unify working + knowledge memory into a vector index for semantically meaningful search. ```redis:[run_confirmation=true] Create a Vector Index -FT.CREATE idx:ai_memory +FT.CREATE idx:memory ON HASH - PREFIX 1 memory: + PREFIX 2 vmemory: user: SCHEMA - user_id TAG SORTABLE - session_id TAG SORTABLE - message TEXT - context TEXT - timestamp NUMERIC SORTABLE + user_id TAG + memory_type TAG + content TEXT + importance NUMERIC + timestamp NUMERIC embedding VECTOR HNSW 6 TYPE FLOAT32 - DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 + DIM 8 // DIM = embedding size; 8 used here for simplicity — in production, use 128 to 1536 DISTANCE_METRIC COSINE // COSINE = measures semantic closeness ``` -Add sample vectorized messages (embedding dims are demo-sized): +### Add Indexed Memory +Populate the vector index with memory items from both ephemeral and persistent layers. ```redis:[run_confirmation=true] Add entries for the chatbot -HSET memory:alice:1 user_id "alice" session_id "morning" message "I have a dentist appointment at 3 PM today" context "healthcare scheduling appointment" timestamp 1717935301 embedding "\x3f\x00\x00\x00\x40\x00\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00" -HSET memory:alice:2 user_id "alice" session_id "morning" message "Remind me to water the plants in my office" context "task reminder plants office" timestamp 1717935361 embedding "\x40\x00\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x3f\x00\x00\x00\x40\x00\x00\x00" -HSET memory:alice:3 user_id "alice" session_id "work" message "Schedule a meeting with the engineering team" context "work scheduling meeting team" timestamp 1717935420 embedding "\x40\x40\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x3f\x00\x00\x00" -HSET memory:bob:1 user_id "bob" session_id "work" message "I need to review the quarterly sales report" context "business analysis quarterly report" timestamp 1717935480 embedding "\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x00\x00\x00\x40\x00\x00\x00\x00\x00\x00\x00" +// Working memory (expires from Redis, stays in search until rebuild) +HSET vmemory:alice:001 user_id "alice" memory_type "working" content "Book dentist for Tuesday 2 PM" importance 8 timestamp 1717935310 embedding "\x3f\x80\x00\x00\x40\x00\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x20\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00" + +// Knowledge base (persistent) +HSET vmemory:alice:kb:001 user_id "alice" memory_type "knowledge" content "allergic to shellfish" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" ``` -### Let Chatbot Think – Semantic Search with Vectors -When a user says something new, find all related past conversations across your entire system based on semantic meaning. +### Full Memory Search +Search across all memories to recall relevant data. ```redis:[run_confirmation=false] Find Top 5 Related Messages By Meaning -FT.SEARCH idx:ai_memory - "*=>[KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - RETURN 4 user_id message context timestamp - SORTBY score ASC - DIALECT 2 +FT.SEARCH idx:memory + "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + RETURN 4 content memory_type importance score + SORTBY score ASC + DIALECT 2 ``` -Now your assistant “remembers” things it’s heard before - by meaning. - -### User-Scoped Semantic Search -Your AI should only recall memories from the specific user it's talking to, not leak information between users. +### Session-Only Search +Limit results to current conversation context (ephemeral memory). -```redis:[run_confirmation=false] Find Similar Memories For Specific User Only -FT.SEARCH idx:ai_memory - "(@user_id:{alice}) => [KNN 3 @embedding $query_vec AS vector_score]" - PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - RETURN 5 user_id message context vector_score session_id - SORTBY vector_score ASC +```redis:[run_confirmation=false] Session-Only Search +FT.SEARCH idx:memory + "(@user_id:{alice} @memory_type:{working}) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + RETURN 4 content score timestamp memory_type + SORTBY score ASC DIALECT 2 ``` -### Time-Bounded Semantic Search -When users ask about "recent" things, limit your search to a specific time window while still using semantic matching. +### Knowledge-Only Search +Focus only on persistent memory: facts, preferences, decisions. -```redis:[run_confirmation=false] Find recent similar memories (last 24 hours) -FT.SEARCH idx:ai_memory - "(@timestamp:[1717849200 +inf]) => [KNN 3 @embedding $query_vec AS vector_score]" - PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - RETURN 6 message timestamp vector_score - SORTBY timestamp DESC +```redis:[run_confirmation=false] Knowledge-Only Search +FT.SEARCH idx:memory + "(@user_id:{alice} @memory_type:{knowledge}) => [KNN 8 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC DIALECT 2 ``` -### Session-Specific Recall -When users refer to something from "earlier in our conversation," search only within the current session context. +### Monitoring Memory State +Use these queries to inspect what’s stored in memory. -``` -FT.SEARCH idx:ai_memory - "(@user_id:{alice} @session_id:{morning}) => [KNN 10 @embedding $query_vec AS vector_score]" - PARAMS 2 query_vec "\x00\x00@@\x00\x00\x80@\x00\x00\x00@\x00\x00\x80?\x00\x00@@\x00\x00\x00@" - RETURN 4 message context vector_score timestamp - SORTBY vector_score - DIALECT 2 +```redis:[run_confirmation=false] Check Memory State +// Check active session memory +HGETALL user:alice:knowledge:pref:001 // Example for one preference item + +// View user knowledge +HGETALL user:alice:knowledge:pref:001 + +// Search user's memories +FT.SEARCH idx:memory + "@user_id:{alice}" + RETURN 3 content memory_type importance ``` -### Want to check what's still in memory? +### Data Cleanup -Only unexpired fields remain. +For privacy compliance, delete all user-related keys. -```redis:[run_confirmation=false] Check Sessions -HGETALL memory:alice:1 -HGETALL memory:alice:2 -HGETALL memory:alice:3 -HGETALL memory:bob:1 +```redis:[run_confirmation=true] Complete user removal +DEL user:alice:knowledge:pref:001 +DEL user:alice:knowledge:pref:002 +DEL user:alice:knowledge:personal:001 +DEL user:alice:knowledge:personal:002 +DEL user:alice:knowledge:work:001 +DEL user:alice:knowledge:work:002 +DEL vmemory:alice:001 +DEL vmemory:alice:kb:001 ``` ### Next Steps Now that your assistant has memory and meaning, you can: - - Combine Redis Vector Search with LLMs (RAG) - - Use OpenAI or sentence-transformers for embeddings - - Add fallback persistent storage with Redis Flex - - Manage users with ACLs, quotas, and keyspace notifications \ No newline at end of file + - Combine with RAG Pipelines + - Use sentence-transformers to generate high-dimensional vectors + - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence + - Use Redis ACLs to isolate users, enforce quotas, and monitor usage \ No newline at end of file From ab887f18e952bedb04e36e548ce5b71bd5e6226a Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka Date: Wed, 11 Jun 2025 13:15:56 +0200 Subject: [PATCH 05/45] Updating tutorial for recommendations --- src/uc/personalized_recommendations.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/src/uc/personalized_recommendations.md b/src/uc/personalized_recommendations.md index 020c4ce..bb53fb6 100644 --- a/src/uc/personalized_recommendations.md +++ b/src/uc/personalized_recommendations.md @@ -1,10 +1,11 @@ -Imagine you’re building a movie recommendation app, and your users expect intuitive, meaningful results, not just keyword matches, but intelligent understanding. - -Not just keyword matching, but semantic understanding — powered by vector embeddings. These are numerical representations of text that capture meaning, allowing you to search by intent rather than exact words. +Imagine you’re building a movie recommendation app that goes beyond simple keyword matching. Your users want recommendations based on the meaning of the movie plots — an intuitive and meaningful search experience powered by semantic understanding. ### Store Movie Documents with Vector Embeddings - -Let’s import a dataset containing plot summaries, each paired with an embedding vector. Each movie JSON document contains metadata like title, genre, year, and a vector embedding of the plot. +Semantic search uses vector embeddings — numerical representations of text that capture the meaning of sentences. This enables search by intent rather than just keywords. +Let’s import a dataset containing plot summaries, each paired with an embedding vector. +Each movie is stored as a JSON document with: + - `title`, `genres`, `year`, `plot` + - `embedding`, which is a binary-encoded `FLOAT32[]` used for vector similarity that can be generated using sentence-transformers or similar libraries. Demo uses 8-dim vectors; production models typically use 128–1536 dimensions. ```redis:[run_confirmation=true] Upload Movies JSON.SET movie:001 $ '{"title":"Toy Story","genres":["Animation","Comedy","Family"],"plot":"Toys come to life when humans arent around.","year":1995,"embedding":[0.22,0.04,0.33,0.12,-0.02,0.17,0.09,0.01]}' @@ -154,7 +155,6 @@ FT.SEARCH idx:movies '@genres:{"Animated"|"Sci-Fi"} =>[KNN 5 @embedding $vec AS This makes Redis recommendations responsive to evolving user preferences without retraining embeddings. ### Next Steps - - Now that you’ve seen the basics, here are a few ideas to extend this: - - Build personalized watchlists based on user preferences and past behavior. - - Power chatbots that provide context-aware movie suggestions. - - Blend keyword and vector search for a richer discovery experience. + - Build a UI that lets users type natural language queries and get semantic recommendations instantly + - Add personalization by combining user preferences with semantic search + - Explore advanced vector search techniques like HNSW indexing for large datasets \ No newline at end of file From 455384c0b02733778702b17fdae074b1519501eb Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka Date: Wed, 11 Jun 2025 16:03:04 +0200 Subject: [PATCH 06/45] Updating a tutorial for recommendations --- src/uc/personalized_recommendations.md | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/src/uc/personalized_recommendations.md b/src/uc/personalized_recommendations.md index bb53fb6..fccbae4 100644 --- a/src/uc/personalized_recommendations.md +++ b/src/uc/personalized_recommendations.md @@ -1,13 +1,16 @@ -Imagine you’re building a movie recommendation app that goes beyond simple keyword matching. Your users want recommendations based on the meaning of the movie plots — an intuitive and meaningful search experience powered by semantic understanding. +Imagine building a movie recommendation app that goes beyond keyword matching. Your users get intuitive, meaningful suggestions based on the true meaning of movie plots — powered by semantic understanding. ### Store Movie Documents with Vector Embeddings -Semantic search uses vector embeddings — numerical representations of text that capture the meaning of sentences. This enables search by intent rather than just keywords. -Let’s import a dataset containing plot summaries, each paired with an embedding vector. +Semantic search uses vector embeddings — numeric representations of text that capture meaning, enabling search by intent rather than keywords. + +We'll import a dataset of plot summaries, each paired with an embedding vector. + Each movie is stored as a JSON document with: - - `title`, `genres`, `year`, `plot` - - `embedding`, which is a binary-encoded `FLOAT32[]` used for vector similarity that can be generated using sentence-transformers or similar libraries. Demo uses 8-dim vectors; production models typically use 128–1536 dimensions. + - `title`, `genres`, `year`, `plot` + - `embedding`: a binary-encoded `FLOAT32[]` for vector similarity, generated via sentence-transformer models or similar. ```redis:[run_confirmation=true] Upload Movies +// Demo uses 8 DIM embeddings; production typically uses 128–1536D. JSON.SET movie:001 $ '{"title":"Toy Story","genres":["Animation","Comedy","Family"],"plot":"Toys come to life when humans arent around.","year":1995,"embedding":[0.22,0.04,0.33,0.12,-0.02,0.17,0.09,0.01]}' JSON.SET movie:002 $ '{"title":"Inside Out","genres":["Animation","Comedy","Drama"],"plot":"Emotions guide a young girl through change.","year":2015,"embedding":[0.20,0.03,0.31,0.11,-0.03,0.16,0.08,0.02]}' JSON.SET movie:003 $ '{"title":"Whiplash","genres":["Drama","Music"],"plot":"A young drummer is pushed to greatness.","year":2014,"embedding":[0.14,0.01,0.22,0.08,-0.07,0.10,0.04,0.00]}' @@ -66,7 +69,7 @@ JSON.SET movie:55 $ '{"title":"All That Jazz","year":1979,"genres":["Drama","Mus ``` ### Create a Vector-Enabled Search Index -Redis stores movie data as JSON documents with text fields (title, genres, plot) and vector embeddings. Creating an index lets you quickly filter documents and run fast approximate nearest neighbor (ANN) searches on embeddings. +Redis stores movie data as JSON documents with text fields (title, genres, plot) and vector embeddings. Indexing enables fast filtering and approximate nearest neighbor (ANN) searches on embeddings. ```redis:[run_confirmation=true] Create a Vector Index FT.CREATE idx:movies ON JSON PREFIX 1 "movie:" SCHEMA @@ -84,7 +87,7 @@ FT.CREATE idx:movies ON JSON PREFIX 1 "movie:" SCHEMA This sets the stage for combined textual and semantic search. ### Semantic Search by Query Embedding -When users search: “I want a fun animated movie about toys and friendship.”, you can generate embeddings using models like text-embedding-3-small from OpenAI, or sentence-transformers from Hugging Face. These models turn text into numerical vectors you can store and query with Redis. +When users search “I want a fun animated movie about toys and friendship”, sentence-transformers models can convert the text into a vector. You can store and query this vector in Redis for semantic search. ```redis:[run_confirmation=false] Search Per Plot FT.SEARCH idx:movies "*=>[KNN 3 @embedding $vec AS score]" @@ -119,7 +122,7 @@ Let’s say you love Inception and want similar movies. Let’s retrieve the Inc ```redis:[run_confirmation=false] Get the Embedding From the Movie Document JSON.GET movie:006 $.embedding ``` -Now let’s run vector similarity search using that embedding as a binary blob (FLOAT32-packed): +Now let’s run vector similarity search using that embedding as a binary blob (`FLOAT32`-packed): ```redis:[run_confirmation=false] Search for Similar Movies FT.SEARCH idx:movies "*=>[KNN 5 @embedding $vec AS score]" @@ -155,6 +158,7 @@ FT.SEARCH idx:movies '@genres:{"Animated"|"Sci-Fi"} =>[KNN 5 @embedding $vec AS This makes Redis recommendations responsive to evolving user preferences without retraining embeddings. ### Next Steps - - Build a UI that lets users type natural language queries and get semantic recommendations instantly - - Add personalization by combining user preferences with semantic search - - Explore advanced vector search techniques like HNSW indexing for large datasets \ No newline at end of file + - Learn more about building personalized recommendations with this [workshop](https://github.com/redis-developer/redis-movies-searcher/tree/springio-2025-workshop). + - Build a UI for natural language queries that delivers instant semantic recommendations. + - Add personalization by merging user preferences with semantic search. + - Explore advanced vector search methods like HNSW indexing for large datasets. \ No newline at end of file From ff4a6738a9937e45a9746fa37becd16213a9339b Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 1 Jul 2025 19:42:10 +0200 Subject: [PATCH 07/45] Update ai_assistant.md --- src/uc/ai_assistant.md | 110 ++++++++++++++++++++++------------------- 1 file changed, 59 insertions(+), 51 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index baee798..d4a4291 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -5,7 +5,7 @@ What you get: - **Semantic Search**: Recall relevant info by meaning using vector search - **Zero Maintenance**: Auto-expiring short-term memory without a need to track timestamps manually - **Multi-User**: Isolated memory per user - - **Learning**: Assistant can "understand" each user better the more it's used + - **Learning**: Assistant learns user preferences and context over time. **Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes, which is ideal for managing short-term memory with precision. @@ -13,27 +13,47 @@ What you get: | Layer | Description | | ---------- | ---------- | | `Working Memory`| `Short-term chat context (ephemeral)` | -| `Knowledge Base` | `Persistent facts, user preferences` | +| `Knowledge Base` | `Long-term memory: persistent facts and preferences` | | `Vector Search` | `Unified semantic recall across both layers` | ### Working Memory (Ephemeral) -Stores recent user messages with TTL based on importance. Automatically expires to prevent bloat. -This uses `HSETEX`, a Redis 8 command that adds field-level expiration to hashes. It allows storing all temporary messages in a single key while managing TTLs per message, simplifying short-term memory management without needing multiple keys. +Stores recent user and assistant messages with per-message TTL. Uses: + - `HSETEX` to add field-level expiration to hashes to store all temporary messages in a single key while managing TTLs per message, simplifying short-term memory management without needing multiple keys. + - `ZADD` for message ordering (based on timestamp) -```redis:[run_confirmation=true] Recent Conversations with TTL Based on Importance. -// Quick exchanges (5 min) +```redis:[run_confirmation=true] Recent conversations with TTL based on importance +// Step 1: Add message with TTL (5 mins) HSETEX user:alice:session:001 EX 300 FIELDS 1 msg:001 "What's the weather?" -// Session context (30 min) +// Step 2: Track order with timestamp (epoch seconds) +ZADD user:alice:session:001:zorder 1717935001 msg:001 +// Another message (30 min TTL) HSETEX user:alice:session:001 EX 1800 FIELDS 1 msg:002 "I need a dentist appointment" -// Important decisions (2 hours) -HSETEX user:alice:session:001 EX 7200 FIELDS 1 msg:003 "Book it for Tuesday 2 PM" +ZADD user:alice:session:001:zorder 1717935030 msg:002 +// Assistant reply (2 hour TTL) +HSETEX user:alice:session:001 EX 7200 FIELDS 1 msg:003 "Booked for Tuesday 2 PM" +ZADD user:alice:session:001:zorder 1717935090 msg:003 +``` + +```redis:[run_confirmation=true] Reading the session in order +ZRANGEBYSCORE user:alice:session:001:zorder -inf +inf +// Then for each msg ID: +HGET user:alice:session:001 msg:001 +HGET user:alice:session:001 msg:002 +HGET user:alice:session:001 msg:003 +``` + +```redis:[run_confirmation=true] ZSET cleanup (recommended in background): +// For each msg:XXX in the ZSET, check if it still exists +HEXISTS user:alice:session:001 msg:003 +// If not, clean up: +ZREM user:alice:session:001:zorder msg:003 ``` ### Knowledge Base (Persistent) Long-term memory: stores important facts, user preferences, and context across sessions. These never expire. `embedding` is a binary-encoded `FLOAT32[]` used for vector similarity that can be generated using sentence-transformers or similar libraries. Demo uses 8-dim vectors; production models typically use 128–1536 dimensions. -```redis:[run_confirmation=true] Important User Information That Never Expires. +```redis:[run_confirmation=true] Important user information that never expires // User preferences - need vector fields for search HSET user:alice:knowledge:pref:001 user_id "alice" memory_type "knowledge" content "prefers mornings before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" HSET user:alice:knowledge:pref:002 user_id "alice" memory_type "knowledge" content "likes detailed explanations" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" @@ -46,12 +66,12 @@ HSET user:alice:knowledge:work:002 user_id "alice" memory_type "knowledge" conte ``` ### Vector Search: Semantic Memory Recall -Unify working + knowledge memory into a vector index for semantically meaningful search. +Indexing persistent memory (knowledge base) for semantically meaningful search. -```redis:[run_confirmation=true] Create a Vector Index -FT.CREATE idx:memory +```redis:[run_confirmation=true] Create a vector index +FT.CREATE idx:kbmemory ON HASH - PREFIX 2 vmemory: user: + PREFIX 1 user: SCHEMA user_id TAG memory_type TAG @@ -64,22 +84,11 @@ FT.CREATE idx:memory DISTANCE_METRIC COSINE // COSINE = measures semantic closeness ``` -### Add Indexed Memory -Populate the vector index with memory items from both ephemeral and persistent layers. - -```redis:[run_confirmation=true] Add entries for the chatbot -// Working memory (expires from Redis, stays in search until rebuild) -HSET vmemory:alice:001 user_id "alice" memory_type "working" content "Book dentist for Tuesday 2 PM" importance 8 timestamp 1717935310 embedding "\x3f\x80\x00\x00\x40\x00\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x20\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00" - -// Knowledge base (persistent) -HSET vmemory:alice:kb:001 user_id "alice" memory_type "knowledge" content "allergic to shellfish" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" -``` +### Search for most relevant memory entries +Find the top 5 most semantically relevant knowledge memory entries for user "alice" by performing a vector similarity search on the embedding field. -### Full Memory Search -Search across all memories to recall relevant data. - -```redis:[run_confirmation=false] Find Top 5 Related Messages By Meaning -FT.SEARCH idx:memory +```redis:[run_confirmation=false] Find top 5 related messages by meaning +FT.SEARCH idx:kbmemory "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" RETURN 4 content memory_type importance score @@ -87,34 +96,33 @@ FT.SEARCH idx:memory DIALECT 2 ``` -### Session-Only Search -Limit results to current conversation context (ephemeral memory). +### Search for High-Importance Knowledge Items Only +This query finds the top 3 most relevant knowledge memories for user "alice" that have an importance score above 7. -```redis:[run_confirmation=false] Session-Only Search -FT.SEARCH idx:memory - "(@user_id:{alice} @memory_type:{working}) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - RETURN 4 content score timestamp memory_type - SORTBY score ASC - DIALECT 2 +```redis:[run_confirmation=false] Knowledge-only search +FT.SEARCH idx:kbmemory + "(@user_id:{alice} @memory_type:{knowledge} @importance:[7 +inf]) => [KNN 3 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC + DIALECT 2 ``` -### Knowledge-Only Search -Focus only on persistent memory: facts, preferences, decisions. - -```redis:[run_confirmation=false] Knowledge-Only Search -FT.SEARCH idx:memory - "(@user_id:{alice} @memory_type:{knowledge}) => [KNN 8 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC - DIALECT 2 +### Search Working Memory Excluding Messages Older Than a Certain Timestamp +Retrieve the most similar knowledge memories for user "alice" that were created after a given timestamp (e.g., 1717935000), ensuring you only get recent context: +```redis:[run_confirmation=false] Session-only search +FT.SEARCH idx:kbmemory + "(@user_id:{alice} @memory_type:{knowledge} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + RETURN 4 content timestamp memory_type score + SORTBY score ASC + DIALECT 2 ``` ### Monitoring Memory State Use these queries to inspect what’s stored in memory. -```redis:[run_confirmation=false] Check Memory State +```redis:[run_confirmation=false] Check memory state // Check active session memory HGETALL user:alice:knowledge:pref:001 // Example for one preference item @@ -122,7 +130,7 @@ HGETALL user:alice:knowledge:pref:001 // Example for one preference item HGETALL user:alice:knowledge:pref:001 // Search user's memories -FT.SEARCH idx:memory +FT.SEARCH idx:kbmemory "@user_id:{alice}" RETURN 3 content memory_type importance ``` @@ -147,4 +155,4 @@ Now that your assistant has memory and meaning, you can: - Combine with RAG Pipelines - Use sentence-transformers to generate high-dimensional vectors - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence - - Use Redis ACLs to isolate users, enforce quotas, and monitor usage \ No newline at end of file + - Use Redis ACLs to isolate users, enforce quotas, and monitor usage From cecbf6be283341ae694df6fd6bd6777402ca9258 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Fri, 4 Jul 2025 16:05:37 +0200 Subject: [PATCH 08/45] Update ai_assistant.md --- src/uc/ai_assistant.md | 254 ++++++++++++++++++++++++----------------- 1 file changed, 150 insertions(+), 104 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index d4a4291..b7d368c 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -1,153 +1,199 @@ -This intelligent AI assistant is designed to support real-world, multi-session use with Redis as its memory core. - -What you get: - - **Smart Memory**: Ephemeral context that expires automatically, long-term facts retained forever - - **Semantic Search**: Recall relevant info by meaning using vector search - - **Zero Maintenance**: Auto-expiring short-term memory without a need to track timestamps manually - - **Multi-User**: Isolated memory per user - - **Learning**: Assistant learns user preferences and context over time. +This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. **Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes, which is ideal for managing short-term memory with precision. ### Architecture Overview -| Layer | Description | -| ---------- | ---------- | -| `Working Memory`| `Short-term chat context (ephemeral)` | -| `Knowledge Base` | `Long-term memory: persistent facts and preferences` | -| `Vector Search` | `Unified semantic recall across both layers` | - -### Working Memory (Ephemeral) -Stores recent user and assistant messages with per-message TTL. Uses: - - `HSETEX` to add field-level expiration to hashes to store all temporary messages in a single key while managing TTLs per message, simplifying short-term memory management without needing multiple keys. - - `ZADD` for message ordering (based on timestamp) - -```redis:[run_confirmation=true] Recent conversations with TTL based on importance -// Step 1: Add message with TTL (5 mins) -HSETEX user:alice:session:001 EX 300 FIELDS 1 msg:001 "What's the weather?" -// Step 2: Track order with timestamp (epoch seconds) -ZADD user:alice:session:001:zorder 1717935001 msg:001 -// Another message (30 min TTL) -HSETEX user:alice:session:001 EX 1800 FIELDS 1 msg:002 "I need a dentist appointment" -ZADD user:alice:session:001:zorder 1717935030 msg:002 -// Assistant reply (2 hour TTL) -HSETEX user:alice:session:001 EX 7200 FIELDS 1 msg:003 "Booked for Tuesday 2 PM" -ZADD user:alice:session:001:zorder 1717935090 msg:003 +| Layer | Description | Data type | +| ---------- | ---------- | ---------- | +| `Session History`| `Recent conversation context` | List | +| `Rate Limiting` | `Per-user request throttling` | Hash | +| `Knowledge Base` | `Long-term facts and preferences` | Hash | + +### Session History +AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated and the AI couldn't reference what was discussed earlier. We can store conversation history using Redis lists - simple, ordered, and efficient for chat scenarios. + +```redis:[run_confirmation=true] Store conversation history +// Add user message to session +LPUSH user:alice:history:session_001 '{"type": "human", "content": "What's the weather like?", "timestamp": 1717935001}' + +// Add AI response +LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It's sunny with 75°F temperature.", "timestamp": 1717935002}' + +// Add another user message +LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bring an umbrella?", "timestamp": 1717935003}' + +// Add AI response +LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' +``` +### Reading Conversation History +Now we can retrieve conversation history to provide context to the AI. + +```redis:[run_confirmation=true] Read conversation history +// Get last 5 messages (most recent first) +LRANGE user:alice:history:session_001 0 4 + +// Get all messages in session +LRANGE user:alice:history:session_001 0 -1 + +// Get specific message by index +LINDEX user:alice:history:session_001 0 + +// Check how many messages in session +LLEN user:alice:history:session_001 ``` +### Session Expiration +Without expiration, conversation history would accumulate indefinitely, consuming memory and potentially exposing sensitive information. TTL ensures privacy and efficient memory usage. -```redis:[run_confirmation=true] Reading the session in order -ZRANGEBYSCORE user:alice:session:001:zorder -inf +inf -// Then for each msg ID: -HGET user:alice:session:001 msg:001 -HGET user:alice:session:001 msg:002 -HGET user:alice:session:001 msg:003 +```redis:[run_confirmation=true] Session expiration +// Set session to expire in 24 hours +EXPIRE user:alice:history:session_001 86400 + +// Set session to expire in 1 hour +EXPIRE user:alice:history:session_001 3600 + +// Check remaining TTL +TTL user:alice:history:session_001 + +// Remove expiration (make persistent) +PERSIST user:alice:history:session_001 ``` -```redis:[run_confirmation=true] ZSET cleanup (recommended in background): -// For each msg:XXX in the ZSET, check if it still exists -HEXISTS user:alice:session:001 msg:003 -// If not, clean up: -ZREM user:alice:session:001:zorder msg:003 +### Rate Limiting +Rate limiting prevents abuse and ensures fair resource usage. Without it, users could overwhelm the system with requests, degrading performance for everyone. + +```redis:[run_confirmation=true] Initialize Rate Limiting +// First request - set counter with 1-minute TTL +HSETEX user:alice:rate_limit EX 60 FIELDS 1 requests_per_minute 1 + +// Check current request count +HGET user:alice:rate_limit requests_per_minute ``` -### Knowledge Base (Persistent) -Long-term memory: stores important facts, user preferences, and context across sessions. These never expire. -`embedding` is a binary-encoded `FLOAT32[]` used for vector similarity that can be generated using sentence-transformers or similar libraries. Demo uses 8-dim vectors; production models typically use 128–1536 dimensions. - -```redis:[run_confirmation=true] Important user information that never expires -// User preferences - need vector fields for search -HSET user:alice:knowledge:pref:001 user_id "alice" memory_type "knowledge" content "prefers mornings before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" -HSET user:alice:knowledge:pref:002 user_id "alice" memory_type "knowledge" content "likes detailed explanations" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" -// Personal facts -HSET user:alice:knowledge:personal:001 user_id "alice" memory_type "knowledge" content "allergic to shellfish" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" -HSET user:alice:knowledge:personal:002 user_id "alice" memory_type "knowledge" content "golden retriever named Max" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" -// Work context -HSET user:alice:knowledge:work:001 user_id "alice" memory_type "knowledge" content "Senior PM at TechCorp" importance 8 timestamp 1717935000 embedding "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" -HSET user:alice:knowledge:work:002 user_id "alice" memory_type "knowledge" content "leading Project Apollo" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x80\x00\x00\x3f\x40\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" +The `HINCR` command allows you to atomically increment the counter, preventing race conditions in high-concurrency scenarios. + +```redis:[run_confirmation=true] Increment Requests +// Increment request counter +HINCR user:alice:rate_limit requests_per_minute + +// Check if field exists and get count +HEXISTS user:alice:rate_limit requests_per_minute +HGET user:alice:rate_limit requests_per_minute + +// Check TTL on the hash +TTL user:alice:rate_limit +``` + +Different time windows serve different purposes - per-minute prevents burst attacks, per-hour prevents sustained abuse, per-day enforces usage quotas. +```redis:[run_confirmation=true] Rate Limiting with Different Time Windows +// Set multiple rate limits with different TTLs +HSETEX user:alice:rate_limit EX 60 FIELDS 2 requests_per_minute 1 requests_per_hour 1 + +// Daily rate limit (24 hours) +HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 + +// Check all rate limits +HGETALL user:alice:rate_limit +``` + +### Knowledge Base (Persistent Memory) +AI assistants become more helpful when they remember user preferences, important facts, and context across sessions. This creates a personalized experience that improves over time. +Remembering user preferences (meeting times, communication style) enables the AI to provide more relevant and personalized responses without asking the same questions repeatedly. + +```redis:[run_confirmation=true] Store User Preferences +// Always secure sensitive data using encryption at rest, access control (Redis ACLs), and comply with data protection laws (e.g., GDPR). +// Store morning preference +HSET user:alice:knowledge:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" + +// Storing communication preference +HSET user:alice:knowledge:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + +// Store work schedule preference +HSET user:alice:knowledge:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + +// Store health information +HSET user:alice:knowledge:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + +// Store pet information +HSET user:alice:knowledge:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + +// Store family information +HSET user:alice:knowledge:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" ``` ### Vector Search: Semantic Memory Recall +Traditional keyword search misses semantic meaning. When a user asks about "scheduling meetings," vector search can find relevant information about "prefers morning appointments" even though the keywords don't match exactly. + Indexing persistent memory (knowledge base) for semantically meaningful search. -```redis:[run_confirmation=true] Create a vector index -FT.CREATE idx:kbmemory +```redis:[run_confirmation=true] Create a Vector Index +// Create index for semantic search +FT.CREATE idx:knowledge ON HASH PREFIX 1 user: SCHEMA user_id TAG - memory_type TAG content TEXT importance NUMERIC timestamp NUMERIC embedding VECTOR HNSW 6 TYPE FLOAT32 - DIM 8 // DIM = embedding size; 8 used here for simplicity — in production, use 128 to 1536 - DISTANCE_METRIC COSINE // COSINE = measures semantic closeness + DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. + DISTANCE_METRIC COSINE ``` ### Search for most relevant memory entries -Find the top 5 most semantically relevant knowledge memory entries for user "alice" by performing a vector similarity search on the embedding field. -```redis:[run_confirmation=false] Find top 5 related messages by meaning -FT.SEARCH idx:kbmemory +```redis:[run_confirmation=false] Find Top 5 Most Relevant Knowledge Items +FT.SEARCH idx:knowledge "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - RETURN 4 content memory_type importance score + RETURN 4 content importance score timestamp SORTBY score ASC DIALECT 2 ``` -### Search for High-Importance Knowledge Items Only -This query finds the top 3 most relevant knowledge memories for user "alice" that have an importance score above 7. - -```redis:[run_confirmation=false] Knowledge-only search -FT.SEARCH idx:kbmemory - "(@user_id:{alice} @memory_type:{knowledge} @importance:[7 +inf]) => [KNN 3 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC - DIALECT 2 +```redis:[run_confirmation=false] Search High-Importance Items Only +FT.SEARCH idx:knowledge + "(@user_id:{alice} @importance:[8 +inf]) => [KNN 3 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC + DIALECT 2 ``` - -### Search Working Memory Excluding Messages Older Than a Certain Timestamp -Retrieve the most similar knowledge memories for user "alice" that were created after a given timestamp (e.g., 1717935000), ensuring you only get recent context: -```redis:[run_confirmation=false] Session-only search -FT.SEARCH idx:kbmemory - "(@user_id:{alice} @memory_type:{knowledge} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - RETURN 4 content timestamp memory_type score - SORTBY score ASC - DIALECT 2 +```redis:[run_confirmation=false] Search Recent Knowledge Only +FT.SEARCH idx:knowledge + "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + RETURN 4 content timestamp score + SORTBY score ASC + DIALECT 2 ``` -### Monitoring Memory State -Use these queries to inspect what’s stored in memory. +### Memory State Monitoring +Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. +```redis:[run_confirmation=false] Monitor user sessions +// Scan 10,000 keys to find user sessions +SCAN 0 MATCH user:*:history:* COUNT 10000 -```redis:[run_confirmation=false] Check memory state -// Check active session memory -HGETALL user:alice:knowledge:pref:001 // Example for one preference item - -// View user knowledge -HGETALL user:alice:knowledge:pref:001 - -// Search user's memories -FT.SEARCH idx:kbmemory - "@user_id:{alice}" - RETURN 3 content memory_type importance +// Get session statistics +LLEN user:alice:history:session_001 +TTL user:alice:history:session_001 ``` - ### Data Cleanup - -For privacy compliance, delete all user-related keys. - -```redis:[run_confirmation=true] Complete user removal +```redis:[run_confirmation=true] Delete user data +// Remove all user data (GDPR compliance) +DEL user:alice:history:session_001 +DEL user:alice:history:session_002 +DEL user:alice:rate_limit DEL user:alice:knowledge:pref:001 DEL user:alice:knowledge:pref:002 +DEL user:alice:knowledge:pref:003 DEL user:alice:knowledge:personal:001 DEL user:alice:knowledge:personal:002 +DEL user:alice:knowledge:personal:003 DEL user:alice:knowledge:work:001 DEL user:alice:knowledge:work:002 -DEL vmemory:alice:001 -DEL vmemory:alice:kb:001 +DEL user:alice:knowledge:work:003 ``` ### Next Steps From 780ea229cc844a7dfcdf884683ed638ae5603c6c Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Fri, 4 Jul 2025 16:14:03 +0200 Subject: [PATCH 09/45] Update ai_assistant.md --- src/uc/ai_assistant.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index b7d368c..2c4f7b5 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -14,10 +14,10 @@ AI assistants need context from previous messages to provide coherent responses. ```redis:[run_confirmation=true] Store conversation history // Add user message to session -LPUSH user:alice:history:session_001 '{"type": "human", "content": "What's the weather like?", "timestamp": 1717935001}' +LPUSH user:alice:history:session_001 '{"type": "human", "content": "What is the weather like?", "timestamp": 1717935001}' // Add AI response -LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It's sunny with 75°F temperature.", "timestamp": 1717935002}' +LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It is sunny with 75°F temperature.", "timestamp": 1717935002}' // Add another user message LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bring an umbrella?", "timestamp": 1717935003}' @@ -73,7 +73,7 @@ The `HINCR` command allows you to atomically increment the counter, preventing r ```redis:[run_confirmation=true] Increment Requests // Increment request counter -HINCR user:alice:rate_limit requests_per_minute +HINCRBY user:alice:rate_limit requests_per_minute 1 // Check if field exists and get count HEXISTS user:alice:rate_limit requests_per_minute @@ -164,7 +164,7 @@ FT.SEARCH idx:knowledge FT.SEARCH idx:knowledge "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - RETURN 4 content timestamp score + RETURN 3 content timestamp score SORTBY score ASC DIALECT 2 ``` From b093e1cdf18fbba632f10ff6fe2ba381d43fe411 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Mon, 7 Jul 2025 14:35:44 +0200 Subject: [PATCH 10/45] Update ai_assistant.md --- src/uc/ai_assistant.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index 2c4f7b5..0bcca16 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -1,6 +1,6 @@ This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. -**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes, which is ideal for managing short-term memory with precision. +**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes, which is ideal for rate limiting to ensure fair resource usage. ### Architecture Overview | Layer | Description | Data type | From 6802a719d514d19c34cdee749a557094b7e18739 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Mon, 7 Jul 2025 17:09:54 +0200 Subject: [PATCH 11/45] Update ai_assistant.md --- src/uc/ai_assistant.md | 96 +++++++++++++++++++++--------------------- 1 file changed, 47 insertions(+), 49 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index 0bcca16..c0f8442 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -1,16 +1,16 @@ This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. -**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes, which is ideal for rate limiting to ensure fair resource usage. +**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. ### Architecture Overview | Layer | Description | Data type | | ---------- | ---------- | ---------- | | `Session History`| `Recent conversation context` | List | | `Rate Limiting` | `Per-user request throttling` | Hash | -| `Knowledge Base` | `Long-term facts and preferences` | Hash | +| `User Memory` | `Long-term facts and preferences` | Hash | ### Session History -AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated and the AI couldn't reference what was discussed earlier. We can store conversation history using Redis lists - simple, ordered, and efficient for chat scenarios. +AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. ```redis:[run_confirmation=true] Store conversation history // Add user message to session @@ -26,23 +26,22 @@ LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bri LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' ``` ### Reading Conversation History -Now we can retrieve conversation history to provide context to the AI. +You can retrieve recent messages to provide context to the AI. -```redis:[run_confirmation=true] Read conversation history -// Get last 5 messages (most recent first) +```redis:[run_confirmation=no] Read conversation history +// Get the 5 most recent messages LRANGE user:alice:history:session_001 0 4 -// Get all messages in session +// Get the full session LRANGE user:alice:history:session_001 0 -1 - -// Get specific message by index -LINDEX user:alice:history:session_001 0 - -// Check how many messages in session -LLEN user:alice:history:session_001 ``` +You may want to limit the size of history to retain only the N most recent items. Use LTRIM: +```redis:[run_confirmation=yes] Read conversation history +LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items +``` + ### Session Expiration -Without expiration, conversation history would accumulate indefinitely, consuming memory and potentially exposing sensitive information. TTL ensures privacy and efficient memory usage. +Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. ```redis:[run_confirmation=true] Session expiration // Set session to expire in 24 hours @@ -59,14 +58,11 @@ PERSIST user:alice:history:session_001 ``` ### Rate Limiting -Rate limiting prevents abuse and ensures fair resource usage. Without it, users could overwhelm the system with requests, degrading performance for everyone. +Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. ```redis:[run_confirmation=true] Initialize Rate Limiting -// First request - set counter with 1-minute TTL +// On first request - set counter with 1-minute TTL HSETEX user:alice:rate_limit EX 60 FIELDS 1 requests_per_minute 1 - -// Check current request count -HGET user:alice:rate_limit requests_per_minute ``` The `HINCR` command allows you to atomically increment the counter, preventing race conditions in high-concurrency scenarios. @@ -82,6 +78,7 @@ HGET user:alice:rate_limit requests_per_minute // Check TTL on the hash TTL user:alice:rate_limit ``` +**Optionally**: if the count exceeds the allowed threshold, deny the operation. Different time windows serve different purposes - per-minute prevents burst attacks, per-hour prevents sustained abuse, per-day enforces usage quotas. ```redis:[run_confirmation=true] Rate Limiting with Different Time Windows @@ -95,39 +92,38 @@ HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 HGETALL user:alice:rate_limit ``` -### Knowledge Base (Persistent Memory) -AI assistants become more helpful when they remember user preferences, important facts, and context across sessions. This creates a personalized experience that improves over time. -Remembering user preferences (meeting times, communication style) enables the AI to provide more relevant and personalized responses without asking the same questions repeatedly. +### User Memory (Persistent Preferences) +AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. ```redis:[run_confirmation=true] Store User Preferences // Always secure sensitive data using encryption at rest, access control (Redis ACLs), and comply with data protection laws (e.g., GDPR). -// Store morning preference -HSET user:alice:knowledge:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" +// These values are stored with embeddings to support semantic recall later using vector search. +HSET user:alice:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" // Storing communication preference -HSET user:alice:knowledge:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +HSET user:alice:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" // Store work schedule preference -HSET user:alice:knowledge:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" +HSET user:alice:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" // Store health information -HSET user:alice:knowledge:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +HSET user:alice:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" // Store pet information -HSET user:alice:knowledge:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" +HSET user:alice:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" // Store family information -HSET user:alice:knowledge:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" +HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" ``` ### Vector Search: Semantic Memory Recall -Traditional keyword search misses semantic meaning. When a user asks about "scheduling meetings," vector search can find relevant information about "prefers morning appointments" even though the keywords don't match exactly. +Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." -Indexing persistent memory (knowledge base) for semantically meaningful search. +Indexing persistent memory (User Memory) for semantically meaningful search. ```redis:[run_confirmation=true] Create a Vector Index // Create index for semantic search -FT.CREATE idx:knowledge +FT.CREATE idx:preferences ON HASH PREFIX 1 user: SCHEMA @@ -137,14 +133,14 @@ FT.CREATE idx:knowledge timestamp NUMERIC embedding VECTOR HNSW 6 TYPE FLOAT32 - DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. + DIM 8 // DIM 8 is only for demonstration purposes. Real embeddings are typically 128–1536 dimensions depending on the model (e.g., sentence-transformers). DISTANCE_METRIC COSINE ``` ### Search for most relevant memory entries -```redis:[run_confirmation=false] Find Top 5 Most Relevant Knowledge Items -FT.SEARCH idx:knowledge +```redis:[run_confirmation=false] Find Top 5 Most Relevant User Memory Items +FT.SEARCH idx:preferences "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" RETURN 4 content importance score timestamp @@ -153,15 +149,15 @@ FT.SEARCH idx:knowledge ``` ```redis:[run_confirmation=false] Search High-Importance Items Only -FT.SEARCH idx:knowledge +FT.SEARCH idx:preferences "(@user_id:{alice} @importance:[8 +inf]) => [KNN 3 @embedding $vec AS score]" PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" RETURN 4 content importance score timestamp SORTBY score ASC DIALECT 2 ``` -```redis:[run_confirmation=false] Search Recent Knowledge Only -FT.SEARCH idx:knowledge +```redis:[run_confirmation=false] Search Recent User Memories +FT.SEARCH idx:preferences "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" RETURN 3 content timestamp score @@ -172,28 +168,30 @@ FT.SEARCH idx:knowledge ### Memory State Monitoring Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. ```redis:[run_confirmation=false] Monitor user sessions -// Scan 10,000 keys to find user sessions -SCAN 0 MATCH user:*:history:* COUNT 10000 +// Get approximate memory usage of session +MEMORY USAGE user:alice:history:session_001 // Get session statistics LLEN user:alice:history:session_001 TTL user:alice:history:session_001 ``` ### Data Cleanup +Remove all data related to a user (e.g., for GDPR compliance). + ```redis:[run_confirmation=true] Delete user data // Remove all user data (GDPR compliance) DEL user:alice:history:session_001 DEL user:alice:history:session_002 DEL user:alice:rate_limit -DEL user:alice:knowledge:pref:001 -DEL user:alice:knowledge:pref:002 -DEL user:alice:knowledge:pref:003 -DEL user:alice:knowledge:personal:001 -DEL user:alice:knowledge:personal:002 -DEL user:alice:knowledge:personal:003 -DEL user:alice:knowledge:work:001 -DEL user:alice:knowledge:work:002 -DEL user:alice:knowledge:work:003 +DEL user:alice:pref:001 +DEL user:alice:pref:002 +DEL user:alice:pref:003 +DEL user:alice:personal:001 +DEL user:alice:personal:002 +DEL user:alice:personal:003 +DEL user:alice:work:001 +DEL user:alice:work:002 +DEL user:alice:work:003 ``` ### Next Steps From 15313616f763a7bbc1b41e65555efc085f4aff64 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 22 Jul 2025 16:15:17 +0200 Subject: [PATCH 12/45] Update src/manifest.json Co-authored-by: David Dougherty --- src/manifest.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/manifest.json b/src/manifest.json index fe0cccc..16347fa 100644 --- a/src/manifest.json +++ b/src/manifest.json @@ -39,7 +39,7 @@ "type": "internal-link", "id": "ai_assistant", "label": "Creating an AI Assistant", - "summary": "An assistant that temporarily retains conversations and understands by their meaning.", + "summary": "An assistant that temporarily remembers conversations and understands their meaning.", "args": { "initialIsOpen": true, "path": "/uc/ai_assistant.md" From df83936fae6a56ca34474df63dc503b39d378252 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 15:46:57 +0200 Subject: [PATCH 13/45] Update vectors-adv-hash.md --- src/vss/vectors-adv-hash.md | 45 ++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/src/vss/vectors-adv-hash.md b/src/vss/vectors-adv-hash.md index e5733cf..e37a754 100644 --- a/src/vss/vectors-adv-hash.md +++ b/src/vss/vectors-adv-hash.md @@ -1,6 +1,9 @@ -This tutorial will demonstrate Redis Stack's ability to function as a vector database and show you how to perform vector searches on hash data using the bike shop use case. Vector search provides for searching unstructured data, like text and images. Vector search is supported for the hash and JSON data types. +This tutorial shows how to use Redis as a vector database with a bike shop example. You'll learn to run vector searches on hash data, which allows searching unstructured data like text and images. Redis supports vector search for both hash and JSON types. -To be able to run these queries you need to have a vector representation (vector embedding) of your text or images. These types of embeddings are generated using a machine learning model. Here's an example: +To run these queries, you need a vector representation (vector embedding) of your text or images. + +**Note:** +> This is a simple example for illustration. Real embeddings can be much larger. For a more realistic use case, see the [vector search quick start guide](https://redis.io/docs/get-started/vector-database/). ```redis A hash with vector embeddings HSET bikes:10000 @@ -12,11 +15,6 @@ HSET bikes:10000 weight 7.5 description 'This bike delivers a lot of bike for the money. The Shimano Claris 8-speed groupset gives plenty of gear range to tackle hills and there’s room for mudguards and a rack too. That said, we feel this bike is a fantastic option for the rider seeking the versatility that this highly adjustable bike provides.' description_embeddings "\xe8\x86\r<\xd6\x98\xcb<\xe8\x7f\x02\xbc\x13\xbf$=!\xa6\xd093\xab\x95\xb1\x86\xbc4\x9f\x9a\xbd\x0c\xdd\x96\xbcl\x18\x97\x91\x1e\x86;\xe8t\x84<\x84u\x86<\xf1`\x8d<\xc2\xf4A\xbcBk\x86=\xd2i\x9e<\x9b\x9b\xa7\xbc:\xb8\xc1\xbc\xec\xb6\x83\xbc\x91xH\xbd\xf6\x94\x9e\xbc%[\xcd<\xe3\xcd\xc7<\xdd\xad!;E\xe6\x9b<\xc9[6=\xd2\xdaz\xbd\x10\xe1\x7f=\xee\x95F\xbc\x97\x8a\x8f\xbb\x0c\xb7c\xbd\xf3;\xe8\xbc\xa7um\xbd5\x9c}\xbb\xcf\xeb\x97\xbb\x05\xac\xdf<\x8d\x14\x19\xbd\xd2L\xd0<\xad\\\xff<\xda\x1e#\xbb\x8a\xec\xc7\xbcF!&=\x90\xe9\x95\xbc\x9d\xbc,\xbdD,0\xbd\xb1\x81\x01=\xd5^\x86<\xdf\xd0\x19\xbd \xa0\x02\xbdX\xc2\x12=-qh\xde\xba\xf77\xce\xbc\xba\x05\x06=\x14_\xa2\xbb0\xa6\x12\xbcpUq=\xe0\xdf\x1a\xbd\x8e\x15\xa2\xbcoK+\xbc<\xc6\xdc<\xfc\x8d\xe5\xbc\xdbr\xbe:M\xba\x0c\xbdY\x97\x04=\x05\x94\xfb;\xaa\xe0\x1f=\xb6%\x06\xbc\xa5i\xa9<\xc2u(:\x87\x7f-\xbd\xa975<\xd9\x17v\xbd1\xef\xd8\xbc\xdb\xe7\x16<\xec\x97/;\xac\xcb\x03=\xfdl\x9d\xbd\x16\xcb\x0f\xbd\xa37a\xbd\nU\xd2<]\x92\x83=\xa2\xc7\x96:\xf4WD=\xf8\xcd0=\xb4\x81(=\xc2\n\xf7\xbc\xb1zC<\x8bN\xcd\xbbg\x05\t\xbb\xc2\x05\x1b=~\x12\xd3\xba^\xbe\xcd\xbbS!t=\x17\xe6\x03=\xc7-1\xbci\x9b\x84\xbc\xd4bD<\xf7*\x13=\x1bV\x9b\xbd\xfdlP\xb7\x1d\x88\x94=$\xd8\x0b\xbd\xae \x98=\x18\x161\xbc\xf1\xab\xd2\xbcAmz<\xd8\xfa\x7f=*$%\xba\x8b6\xbdD\xa2\x1c\xbd\xa3%\x85=\xf9\x86\xbe\xbcH\xec\xf1;\x96\xb5\xaa<\xdccI\xbd\x16\xfa@=\xac;\xed\xbc\xf9L\xa6\xbb*\xc8K\xbd\xeb5\xe9\xbc\xa8\xe87=\xb6\x02y\xbd\xe3\x0f\xae\xbc\x01\x03\r=\x14pO\xbd\x94\x1e\xbb<\x19\x02L=\xdaQ\x05;\\\xa9D\xbb\x0e\xca\x9d\xbcF\x19X=\xc1\xc8\xec:4\xd9\xb4<\xa5\x0e\xe3<&^\x98\xbb\xd4\x97D<\xf2\xfd\x83=\x0c\x10\x13\xbc\xf5[[\xbd\xbex\xe7;\x9e#4\xbd.\xe7\x80\xbc\x87\xca\x91<\xaed\xf8\xbc\xb1\x0e\xda\xbc\x89\xc6\xb2=W\x82p<\xcb\xaf\x9d\xbc\xbd\x058\xbd;\xa4g\xbdY(l\xbd?Q\xb1\xbdM\xd6#\xbd\xcc\xd2T\xbb\xbc\x136\xbd\xb9\xa3A=\x04\xf3\xe0<\xc8K%=\x88\xf1\x90;L\x15\x8b=\x80\xd62\xbd\x08\xb3J\xbd\x18\xd8a<@\xd0\xca:Y8\x7f<3\x14\x90\xbd\x9d\x03\xd6<\xd4U}<\xf8Br\xbc\x9cd\x95\xbc\xbdO\"\xbd\xff\r\xb1\xbdLL\xa5<\xec\xc2+\xbd:\xda\x1f=\x95\x00\xef8\x0c\x9aH=;D2;\x02\x06\x1d\xbdw\x83\xb8<\xbf\xd7Q<\xe3\x81\x07\xbb?}\xac\xbc\xd4F\xbb=\xdfY\xbb<\xa0\x02\xd8;\x97\x95d=\xb5\xc3\x12=A\xdd\xb8\xbc\xbc\x99\x97\xbd\x0c\xb0\x81<\xb0\xe9f\xbc\xbd\xdcR=Q\xc5l<\xa5\xf7\xc3\xbcn\xe8C\xbc\xdf&]\xbd\xb6\x1b5<\xc4\xad\xc5=\"\xe4\xb9\xbd\xe9\xbdt9VS\x92\xbc\xf90\xa4=uG\x81\xbc\xf4\x83h=>M\x16\xbc\xf0\xc0j\xbd\xc2\xdfH\xbd\xf4\x08}\xbd3lC=\x05(\x1f\xbdb\xfc\xbd;=O\x1c\xbd1[\xad<\xcf\xdc\x84\xbc\x10\"\xfe<\xf9c\xde\xbcZ\x8f\x0b\xbdb\xc8:\xbdN\xb1\"=\xd8\x84\xd0\xbc\xac\x0e5\xbd\xa7\xacL\xbdRr}<*8*\xbdx9Y=\xd0\x87\x9b=\x97\x87\xa1=\xf2D\xd6\xbc U1<\\\x9cf;7\x91\x89\xbb\x9b\xd7D=c\x08\xa3<\x9d\x16\x01\xba\\M\xa1\xbc\xcf\x05\xae\xbc\xd6UK\xbd\xb2\xb8\x95;\x8a]\xa2;\x9c-\x0b\xbd\xf1|8=7\xb5\xb9=}\x81\xa5\xbd]\xbf\xba< \x93\x97\xbaV\"\xfb{<\xd8\xd4\xd3\xbcv\x88\xb3;\xc1\x19\xd9;\x16\xbd\x95=@\xcb\xcf<\xe8\xeez\xbc\xfbRZ\xbd\xae*);\x8e\xb05;h\xaaW\xbc\x98\\\\=\r7\xd4<\x0c\xd2\xb0<\xa9?\x8b<\x05\x1eo\xbd\xbd{\xc1\xbc\x15fm\xbd\x00\xe3\xae<\xe2+\x98<{\x07A:\xd9\x8a[\xb3\xd5<\xd6W\xbd\xba\"\x9c\xa5\xbc\xed\x00M=\"\xc3\x89:\x93w\x14\xc4\xb6\xb9UM\x8a;)Q\x8e<\xd1F(\xbcz\x9e\x03=\xb4\xe2\xf2\xbb\x94\x99$<~\x11>\xbd\xe7i\x15\xbd\x8a\x01\xd6<\xd4\xe9\x9f<`\xbf6\xbc\x08\xd4\xab\xbb\xf8\x0c\x89\xbd@\x9d\t=(\xe5\x0e\xbc6\xd3\x00\xbd\xa8\x82\xa9<{\xd8\xfc\xbc\x81S\x89<\x87\x1f\x17=\x17:\xd0;\xce\xe6]=\xb5\x95\x82\xbb\xf50+\xbcm\xe9n\xbdIn.\xbd\xb4\x95\x01;p\x89i=,\x0b}\xbc\xde\xae\xce\xba\x01\"\x8b<\xb1\xd2(\xbd\xe8\xb6\x1d=\x1ak\xd4\xbc\xb8\x1e\xd4;_\xd0\xde;\xf8\x98\xc4\xbaz\\\xec<\x8c\xd6K<\x94\x84\x17=\x7fv\x91\xbd\xcb\xe7n\xbdP<\xb4\xbd\xf8\xc5\xbd<\x00\x0fF=\x85z\x15<\xc54\"<\xd8\x7fh\xbc\xde\xff\x01\xbca\xf4\xa8\xbc\x94]\xb0\xbc\xe8\xda\x80=?\xa9\x17\xbb\xff4\xc4\xbb\x8a\xf7\xb2=^\xd00\xbd\xa7\xad3\xbd\xf1\xf9\x84:\x1b\xbc%\xbd(\xe84\xbd/\x01\x9b<\xf716=r\x86\xa3=}n\x0c=\x05\xa8\xfd\xb8\xd5\x84C<\xf9\\\xba<\xfe\xb3\xa8\xba\xa9\x82\xaa=\xf6{\x06;\x9eXk=\x86F]=\x18\xa7\xed\xbb\xc1\xef?\xbdXi\xb9:w\x95\x8b\xbc\x0eD\xb3\xbc\xa8\re<\xd7G\xb6&\xbdj\xbc\x9b\xbc5\x02\xa6H;@\xa5.=\x12\xf3=\xbd\x97\xc7D;\xc3U\x8b\xbb\xf0\x04\xdb9!\xeb\xae\xbb\xbc$\x0b\x1e\xbc\"\xe5c=4T\x1c\xbd_\xc9\xec\xbcG8\x85<+\xa2\x91\xbcu\x12\x84\xbd\x99\xce\x02=D\x16R<\xeaS ;\xae\x9e\x1c=\xa1:\x01\xbd2\xc15\xbc\x88u\x8d\xbd\xfd\x85i\xbd\xcb\x00\n\xbdb\x81u\xbc\xd2\x16M\xbcN\xd8\x9e<\xf0\x94\xf2\xbcn\x80\xea\xbcF\xe9\xb0\xbcL9h\xbd\x88;\"\xbd\x8bf%<\xe5\xd6 <\x820!\xbd\n@\x93<\x14R\x91\xbd\xed\xb3j;\xe7\x9d\x8e:\x9c\xee\x07<-T\xd8<\xb2\x96B\xbdb\x14\xa5\xbb\xd0\xd2P\x97\xbc\xaf~\xa4\xb9\x9a\x0b\x1a\xbd\x06\x9c\xd9\xbcX#\"\xbdD\x07\x02=:;0\xbc\xa7v\xaa\xbc\xafE\x97\xbd\xf8\x14W<\xcf\xdd#<\xee\xab\x9a\xbd\xbaw\x17\xbcE\x08 \xbdp\x13\x9d\xbbG,(\xbd\xe6\xdad;\x19\x9f\xa8<%*\x84=\xe2(\r\xbc\x16\xf4\xcb\xbc\xe5\x06,<\x89\xbc^\xbc\x96|\xb0\xba\x1b\x1a\xa7W\x92\xbc\xf8\x99M\xbc4\xfb\xe7\xbc_\x02.<(\xb9&\xbdJ\x97\x8b=\xc6Il\xbc!x\xfb\xbc\"\xc9\xe8<\xabrw<\xf4Ls<\xf0\xc8#=*Tx=M\xdc\xce\xbc\xf3\xca\n=\xc2\xfc\xa5<\x96d\x81<\x86\xd9\"\xbd\xebh\xc8\xbc\xc8\\\x0f=\xc5\x90@\xbd\x07\x1fB<\x91\xc2\xa9\xba\xb75\xba;\xc2*\xc7\xbc\x95\x19\"\xbd\xcf\x0b`=\xdf\xeeP\xbc\xba\x83\x03=\xe8\x01\xe1\xbb\xabdn=#\xe4\xef\xbb\xc1\x94\xb8;\xa5]J\xbd\xcc.\xbf9\x18c\x0b\xbd\x8f\xa6\x06>E\x171\xbcr\x0bz\xbb\xc0;\xe3\xbbio\xc8\xba\x05q\x9b<\n-G\xbb\xdel\xf4\xbc\xe7\x91\xae<\xa0\xc9\x08\xbc\x9cAu\xbc0\xe0\x11;\x00\xb0i\xbc\xf9\tm\xbbF\xb0\xc2\xbd\x90\xfbd\xbc\xe3w\xc9<\x1a\xe1\xd0\xbbD\x0bC=\xe1.f<\xefY\x9a\xbc\x18\xe9\xbd\xbcBbp=C\xfb\x9d\xbd\x84\"\xd8<0\xee?\xbb\xc1\x9e\xad\xbc\x93\xa3`=\xb3D1<&\x17\xaa\xbc\xd7\xa3,\xbb\xc0\x07\xdc<\x8a\xa5\xd5<\xd67\xa8<\x1a\x80d\xbc\xa8\x0b\\\xbc\x8d\x00(\xbc2\x143=\x1f\x07\x08\xbdee/=\x8at\x00=\xab\x04\xaf<\xcblP<63v=\x15\x9c>\xbc\xe9<\x89\xbdli\xd3\xbcmY\xae<\rl\x83<\xb1\xcf\x9a\xbbjn\x93\xbd\t\xe7H=\xe2Cr=\xf2Ai\xbc:\x1a\xc5\xbc\xb7\xc07\xbd\xeb\xa2\x92=Iij;\xa2\x8fp< \x95\x03=\xbd\x14\xa8=\xb5\x12\x92<\x0e\xb4\x9f=\x81\xbe\x1b=\xbaJ\xdc<(\x81\xb0\xbc;\x7f\x9b=\xc3\x8bs\xbb\x03\x81\xa6\xbdp\xc9\x03\xbc\xc1\xdf\n\xbcx\x7f8\xbc{\xc6\xdd\xbcb1\xc8\xbd\xdd&\xe0\xbc\x0c\xea\x8f<;]\xc8<\xe5K\x1d=\xf5\xdc\x83\x8d;\xc2\xd18\xbd\x91\xf4k\xbd\xec\xa4\x17\xbd\x02\xcb8\xbc\xe2bf\xbd\x17\x08\x12=\x17#`;3\xdd\x9d=XF\x89<>\x14\\=\xbb\"@\xbdXH\xf5;\xa4O\xc3<\\\xfbM=\xf9K6\xbd\xb9\xcb\x1f\xbd\xccdK\xbbHi\xd4<;\x04\xea9\x1e\x93\x06\xba\"\xba/\xbc\xa3\xce\xe3<\xc1r[\xbc~\xb2\x7f\xbc\x18\xd3\xb8<\x9d\xab`\xbc\xa0y\x14=\xc7+\xae<\xef|@=\xff\xa9\x9f\xbd2\x08 8\"\x11W;mI6<\xd8\x19\x1f<\x94\xf4\xae<8\x03,;r\x8dm\xbcu\xact\xbc%\xfc\xed\xbb0\xdb\x0b\xbd4o\xf3\xbbw/h=\xa2\x08\xd1\xbc\x96\xcc\x93;mZ\xdc\xbc\\!?\xbb\xd6\x18s;\xfa\xe8\xad<\xd4.\x98=\x1e\xae\xb2<\x844\x9d:\xe55\x8a;\x1a\xfb\x15\xbd\x8a\xd4\x0f\xbcX\xa5Y\xbd\x021\x18=WX\n\xbd;\xb4!\xbd\xe4r\xb6\xbb\xc7\x1a\x7f\xbc\xf4C:\xbd\xed\x9d\xa0<\xbd\xcdH\xbc\xaa\x15\x84\xbc\xf7$L=V\xc32\xbdos\xcb\xbc>\xfc\xac\xbd\x15\x8eX\xbd}\xe1\xfd\xbc\x1b\xc2\xba\xbcsH\xf9\xbb9\x8c\xb6<\x1f\xd5&\xbd\x0b<\x83\xbc\xfd\xf5\xd0\xbc1l\x95\xbd\xda\x7f\xef\xbcU\xa0\x1f<\x14y?\xa0\xbd\x7f\x96\xc1;#\xd9\x8a\xbb\xc6\xfb\xa3<\x88\x80\xbc<\x9d[\n\xbd\xc1\x01\xd8\xbc\xd8\x8eK<\xcdM\xa2\xbc\xe4T\x90\xbd^4\x9c\xbd\xcf\x14P\xbdd\x18M;\xd5\xa8{=,/\x00:T\x92\xb3=\xe1\x13\xe2\xbbm\xdd\xb3<\xb8\x1a\x82\xbd>o\x8d\xbd\xca\xa8)=\xdfD\xf7\xbb[o\r;1\x9dx\xbc\x08\x1c\xb6\xbc\x036f;\x8d:\x93\xbd8=\xc7\xbc\x9b\xaf\xfe<\x1e<:=N\x03\x8b\xbc\t \x83\xbbZ\x89e=\xf7B?<\x1c^\xcc<\x8b\x9do<\x1bE4\xbaf\x0b\r\xbdJp\xa5\xbbT\x12\x88\xbd\xea\xa6\xe6\xb8\xf2\xb5\xdd<\x83\x7f\x11\xbc\x7f\x05V\xb0\xbc\xdb\xb52=\xe5\xc3m=\xf7\x9c2\xbc\xce\xb4\x10=\x06rG\xbd>c\x0f=\xcd\xc5\xf9\xbc" HSET bikes:10002 model 'Tethys' brand '7th Generation' price 2961 type 'Road bikes' material 'alloy' weight 7.4 description 'The bike has a lightweight form factor, making it easier for seniors to use. At this price point, you get a Shimano 105 hydraulic groupset with a RS510 crank set. The wheels have had a slight upgrade for 2022, so you\'re now getting DT Swiss R470 rims with the Formula hubs. All in all it\'s an impressive package for the price, making it very competitive.' description_embeddings "j\xa1\xa5\xbcM\x18\xa1<\xfc\x7f\x9a\xbcV\xb6\xcd3\xe90\xbd\xea\x7f\x97\xbc\xc9\xcdm+\x0c\xbd\xd7\x10\x9f-?=\xda\xd5\xad\xbcE\tB<\x9cR\x13\xbdo\x8e\x0b\xbc\x1e\x86\xb5<\xd98e\xbd\xe9\xed]=lZ\xc7\xbc\x0c\x8d\xd5\xbc\x1c\"\x90\xbd~/\xbf\xbc\xe0U\x87=\xa8\x9d\xb0\xbb\x08`B=F\x19\x02<\xbcYD=\x00Oe\xb9\x10^\xe9<\xbec\x08\xbd\x89U*\xbcr\xd4\x01=\xb1s\xde\xbbB@\xad9T5\xa8=\xafz_\xbc\xaem\xd8\xbd\xady\x96\xbdm\xe2\xa9<\xa51\xaf\xbc\x06\x9a\x93\xbc\xfe5m\xbc;\xdf\xce;\xa8\xdd\xc7\xbd\xdc\x06\x8f=\x9d\xf2(\xbd\xbdf\xfd\xb9Y\x04\x13\xbd\xea\xbcx=,\\\xe59\x1d\x80\x13\xbc~|\xf3\xbc\xe3\xc1\xf1\xbc\xb9>\x97\xbcLL\xa9=\xc6H\x00\xbdq\xd4\xe3\xbcR~\xb2<\xa0Q[<\xb3\xff\xe2;\xb1OP\xbc,9^\xbc+\x1c+=~\xbb\xdf<3R\r\xbd\xcc4\xab<\x99\xa5\x80<\xa1,\x96\xbb\xb49 <\xff\x9d\x98\xbc\xe1=\xdd<\x84\xa45\xbd\x96\x1e/\xbd\xd8\xca:;G\xc3n<\t\xd5v<\x81\x9b\x89\xbd\x84\x1bK\xbbq\xe6\xca<\xfa_\x17=\xf5<\xe6\xc4\x85\xfe;\x1a\xcd(;lH\x1e\xbc{\xfd\t\xbd\x89\x0c\x89\xbdgw\x1d=\xee\xd2^\xbc\x1c8\x85\xbb\xecn4=l\xc3&\xbd\x96\xc2\xbf:}uv;.O\x00=?\xa2\x91<\x1e\xad\n\xbb\x1f\xa5\xbe\xbc\xf9\xc9\x16=\x83\x14\x16\xbb\x85\x89\x84\xbd\x13\xadH\xbc\rL\x97\xbc,p\x94<\x93$\x8b\xbc\xc4i\xb8<\xd5@\x16\xbd\x1cC\xdc<\xb6\xcf\xdf<\x1fMU<\xe6\xef\xc0\xb9Q\x85\xef<$\x85t=\x86\x97\x91\xbc\xcc\x1d\xfd<\xe1]%\xbd?\x92t:\x8f\x1bJ\xbd.\x0cz=-\xbd\x7f\xbdMZ&=\x93?`=kTh=\xbdW1=\xed\x9b\xbd\xbb\x0b\x81\x80<\xe5\x11X\xbd\x0f5C=\x04\xe0&<\x1fn\x1b\xbdU\xf4\x03\xbd;L\x93\xbdD\xa7Q=1\x1dz\xbc\xa8\xd5%= \xafU\xbc\xff\x1a\"=sND\xbd\x9dh.\xbd\x86\tI\xbd/\x88\xef;\xe6\x06\xb2\xbd\xc8\x1c\xa0n\x91\xbc\xcb)U\xbd\xbc\x85N\xbd\xdb\xc8\x9e<\xb0e?\xbd\x9d\xcc\xce<\xe4)4=\xc0\x15;=\x02\x8b\xa2;\xa8\xa9N=Kj<\xbdT\xe2\x8c\xbd\x19\xff\x19=\x93\xcc\xfa:J7\x97<^\xab9\xbd\xd09k;P\x11\xe5<[\xa0e\xbc~Yu<1j,<\x05]\t9E\xe5\xc3\xbc[p\x0c:\x9aSd\xbc\xb0D\\\xbc\x12\x8d0\xbb\xfe}Q<\x7f\x80\n<" @@ -129,25 +127,30 @@ HSET bikes:20009 model 'Secto' brand 'Peaknetic' price 430 type 'Commuter bikes HSET bikes:20010 model 'Summit' brand 'nHill' price 1200 type 'Mountain Bike' material 'alloy' weight 11.3 description 'This budget mountain bike from nHill performs well both on bike paths and on the trail. The fork with 100mm of travel absorbs rough terrain. Fat Kenda Booster tires give you grip in corners and on wet trails. The Shimano Tourney drivetrain offered enough gears for finding a comfortable pace to ride uphill, and the Tektro hydraulic disc brakes break smoothly. Whether you want an affordable bike that you can take to work, but also take trail riding on the weekends or you’re just after a stable, comfortable ride for the bike path, the Summit gives a good value for money.' description_embeddings "\xd2Tm\xbbJD\xbd:r(\x1e\xbd4s\xea<\xcagE:\x00\x83\xd6;\x9e\xef\x80<\xbar\x95\xbd\xce\x14f=\x96\x85\xd4<\xf2|X\xbcM\xf0\x19\xbd\x90\xd1\xee<\xeau\xf1<\x18t\x9d=\xb0\x8b\xcd:\xb9\x0b\xab<@\xf3B;r});\x8c\xaf\xa3\xba\r\x98\x03\xbcHVS\xbaX\xe9\xc6\xbc\xbc5\x8e\xbd\x01w\n\xbd\x939\xbe\xbb\xf5?@=M2r\xbd}L\xbf\xbcT1S\xbb\xa2\xd7\xa7\xbc]\x96\x8b\xbcW\xfd\xf0<\x10A\xaa<\xc5\r\x08=\xdek\x1e\xbbz\xf2\xe1;c\x83\x89;\xfd\x06l\xbd4\xd4\xeb<\xe7\x16\xe6\xbca\xd8\x05\xbc\xd4\x9d\xea\xbb\xe5\x04\xc9\xbbX\x9b\xdc\xbcE\xbcD\xbd\"\xcaF=w\x0e\xbc;\x06\xd7\x1d<\xad\x9d\x99\xbb)\x9e\x83<\xb12?=\x82\xdb\xb3\xbc\x10v\xba;\xba\xf8 =\xf2*\xeb<\x94\xedn\xbbMk\x1d\xbd\x13\x99\xb9\xbc4\xb3\"\xbd\xe9\xfb\xc1\xbbD\xd1\x80\xbc\x83E\xe3\xbcd\xa4\x84\xbb\xe0s\xef\xbc\r\xe0\x88\xbc]A%=\x97\xaa\xab<\x06}\xef9\x03\x98\x1d\xbd\x03\xc9\xdb\xbc\xa4&\x8d<\xdf\x8d\xac\xbc\xe4\xdaw=@\xd2\xa0\xbc\x03\x9a{\xbd\xf1\xb9\x9brY=:\xdd\x87<\xc9\xbfX==\x90\xdf<\x0c[\x1d=\xe1\xa8\xcf\xbc/`\n=\x1b\xd5\x87\xbcSX\x92\xbd\x0f\x89\x82\xbd6\xc4\x06=9\xd6\x92\xbd\x8dzJ\xbc\x0cj\xac\xbc\xae+\x96=\xc3\x8e\x95\xbc\xfc\x8cw;\xaf\x9e\xa5\xbd^\xcek\xbb\xd6\xafo;\x80H\x84<\r\x1c\xb1\xbc\xf6\xef\xe3\xbc\xb0\xfe\x02=\xd6#\xe2;;\xed*<\x04b\xf4\x06\xc2<\x81\xed\x88\xbd\xd1\xcf\x1e\xbc@\x01b\xbb\xc6\xa7\x0b\xbd\xa8\xf2\x85<\xfc\xc8\x97\xbd\xbf/M<\xb4\xb9_=c\xa9\x81=\xd3\\u\xbc,3O;\x1b7\xb4;\xd9P\x1f=\x8b+:=xOf\xbd\xb2\xb4\xc1\xbcb\xe9\x1a\xbc\xae\xac\xda\xbc\x8d<\x95=\xa3\x07(<\xa7\x16\xb6\xbb\t\xab9\xbdx\xb1\xdb\xb9\xf6\x85\x0f=\x16H3\xbc\xe1M\x14\xbc\xd7~\x9d<\x88\x9d =\xf8C\x86=b\x8c\x98\xbb\x05\xf6\x05=hw\xfe\xbc\xf1\xaa\x01\xbd\x13\xd4\x10=\xe0S\x96<\xbf{\x1d\xbc\x18O;=T\xbbA;v\xae\xd1\xbcnL\x11\xbd\x06\x1d\xd7\xbdS\x86\xcf\xbc\x06\xaf\x0f=i\x98\x1b\xbcp\x99\xc4=\x9ebI=\xf5\xd8q\xbb\xfb\x83\x97<\x19\x0e\xaf<|X\x8a\xbd\xa9%\x98\xbc\xdb\xc3\t=P\xb8V=\x99\x82\x16\xbb\xb2\x16\x84;\xf0<\xa1<\xd04\xb3<1\x8d\x19\xbc\xfa\xdaz\xbdC\xd2\xca\xb9[\xae\x84\xbb0\x84+\xbd\xe6\xc0F\xbd\x0b|9\xbc\x7f\xd9o;\xb1\x98H\xbd\xacd\xb95\xc3;\xbd\x8e\xb48\xcc\xb7\x80\xbdm\x9e$=\x18\x855<\xc6r\xb4\xbc\x12\x1d|<\xe8\xec\xc9<\xd7\xdbG\xbd\x16\xf0\x95\xbdu?\x12\xbd\xe6L\xa9=\xbf$\x85P\xc1\xbckS\x91\xbc3\xff(\xbc}\x93\x7f\xba\xa3v\x19\xbd\x1a\xbd\xdf<\xf2\x12^\xbc\xb7\n\x0e\xbc\xecM\xf2<1PU\xbbb\x81\xa3\xbd<\xd8\x19\xbdf\xca\x14<\x87\xddW\xbc\xe5\xd0\xc5\xbb0j\x15\xbd#\xe4\xf7<\xfdL\x1f=\x1b\x00\xbd\xbc\x16\xf8m\t\x92\xcc\xa1\xbc\xf3\xe9\x04\xbd\xfa\xaaL=\x85\x07\x8b<)j\xe9\xbb\xbd\x90 =\n\xf4\x07\xbc\xf2\xf0\x8d<\x82=8A\xdb<\x1b\xed\x0f\x01\xbbd$\x9b\xbdp\xc1p=\xdf\xf0\\<\xdc\x1fK=\x14\xe3c<\xb3\x14\"=\x10\xff\xb4\xbd\x81\xf5\x00\x03\xbc\x04na\xbc_\xd4\xde<\xec\xd5\xcb:\xd2\xba\x06:\xb61\x83\xbb\x01V#\xbd\x98\xe1\x90\x9d;\x8d\x8f\x06=}\x9f4\xbdF6+=\\\xe9S\xbd\x9co+=x\xa3\x0c\xbdW\x8f8\xbds\x97\xaf<\x9d\x0e\xa9\xbcj\xae\xd9;QM8\xbdz%\xe2\xbd\xf0\xa9\x91\xbcp\xbc\x8b\xbcX=\xe8<\xc1n\xce\xbc\x82\xd3\xcb<\t\xa4\x12;\xa34o<\xcb\xdfL<\xcbU\xa2<)S\xf7-=\x94\xc3\xd1\xbd\xf3\xf3\xcaj\xbbs\x9cR\xbd\x9a%\xc0\xbd\xff\xf5\xf6<\xdb\xc3\x13=\xe5\xc2\x1a=\x9c\xa8\"\xbd\xf5\x0b9\xbd\xa5t\xe7=\x94\xc5\xcf=R\x00\xad;\x1c.2;\x14 \xaf\xbc\xbf\xb1\x81\xbd\xc0e\x06\xbd\x1bT\xc4<\x8b\xc0\x8b\xbd\xab\xc6\x9f\xbc\x10\x92\x0f\xbdUP6=&P\xd6<\x0c\xef\n<\x88\x83\xc7<\xbar\xf9<\x9cl\x95;\x9d\x0c\"=e*\xc4;!\xcd\x12=\xac\xae,=\xd1:\x05=0\xf2|;S\xe4\xff\xbb\xa9M;\xbc_\x899\xbd\xf4\xbe4\xbd9\xb1\xda\xbb\x91%\xc5<<\x9e\xad\xbc\xed\x80S\xbdbHp<\xe5\x91\xdc;h\x9b\x12\xbbv$#<\x10\xd0f\xbdocU=d\x17\xe1\xbc\"WC\xbd\xed\xdb \xbcJ!\xa4<\xb4\x7f5\xbd\xb9\x00\x98\xbdAr\x96\xbd\xe6\x18\xb6=:\xf1\xb9@\xf9\xbb\xb7\xa62=\x9a\x03~<\x16[\x99\xbc\x806\x87\xbd\x80\xcf==d\xf9.:\xd8\xfd\xb9<\x9a\xea{\t\xf9\xads\xbcx\x93\x14=\xcbB1\xbc/\xf8\xf5:\x03\"\xd3<\"\x912\xbc\x00+\xf1;\xee\xc4\xf67\xd1\xf4\x14=\xe8\xbcR\xbd\xc6\xb9\r=z$\xd0;\xea\x99A\xbd\x91d\x06=f1\x11\xbd\xd0_+:\x14J\x96\xbc\x87\x87\xc6\xbce\x13.?\xc2\xf1\xbc\x8d\xf6\xf2<\x97hf<\xef7\xf2;\x0e\xfd\xbe\xbc\xd9\x88\xd0\xbc\xfc\xc8\xc4<\xe7D\xc6\xbb5\x0b\x9e=\xc8\xac\xcb<+\x06\n=\xf1\x89\x93\x0b*=\xea\xc5\xd5\xbb\xf3\xc5\\\xbc\xd2\x0eT<\x9a\xc7\x1e:\xbf\xccE<\x00\xf5\xe6<\x94\x8a\xf5\xbbxl\xa7<\xb2A\"\xbc\xc9\xdb\x02\xbc\xdf\x7f\x12=.:s\xbc\xc7\"v<~v\x83\xbb2iF\xbc}\xd5\x1a\xbd\x10\"\xba\xbc^\x92\x1a\xbc\x11\x02\xcf<4:1<\x86\x1b\xae=\x83\x19\xf0 Date: Thu, 24 Jul 2025 15:47:18 +0200 Subject: [PATCH 14/45] Delete src/vss/vectors-basic.md --- src/vss/vectors-basic.md | 82 ---------------------------------------- 1 file changed, 82 deletions(-) delete mode 100644 src/vss/vectors-basic.md diff --git a/src/vss/vectors-basic.md b/src/vss/vectors-basic.md deleted file mode 100644 index 00f4678..0000000 --- a/src/vss/vectors-basic.md +++ /dev/null @@ -1,82 +0,0 @@ -The following JSON data model will be used in this tutorial. The data represents user preferences that a hypothetical bike shop might use to target ads to its customers. - -``` -{ - "user": "morti", - "descr": "Morti is into kid's and road bikes.", - "labels": "kids, road", - "vector_embedding": [ - 0.1, - 0.9, - 0.7 - ] -} -``` - -The `vector_embedding` element is a vector, a list of numbers, that represents preferences for certain kinds of bikes. -Each three-element list represents a preference score for: - -- mountain bikes -- kid's bikes -- road bikes - -**Note:** -> This is just a trivial example that was created as an illustration. Vector embeddings for real projects can be huge. For a more realistic example, see this [vector search quick start guide](https://redis.io/docs/get-started/vector-database/). - -To conduct vector searches, you must first create an index and then load your data. - -```redis Create an index -FT.CREATE idx:user_prefs // index name - ON JSON // the type of data to be indexed - PREFIX 1 user: // identifies the keys to be indexed - SCHEMA - $.descr TEXT // Allows full-text search queries - $.labels TAG SEPARATOR "," // Allows exact-match queries, such as categories or primary keys - $.vector_embedding as vector VECTOR HNSW // Hierarchical navigable small worl vector - 6 // Six parameters - TYPE FLOAT32 // Each vector element is a 32-bit, floating point number - DIM 3 // Each vector has three dimensions - DISTANCE_METRIC COSINE // See the docs for a description -``` - -This command will create an index called `idx:user_prefs` on all JSON data with keys prefixed by `user:`. -The index schema has the following attributes: - -- `$.descr TEXT` - the key's `descr` field, indexed as `TEXT` -- `$.labels TAG SEPARATOR ","` - the key's `labels` field, indexed as `TAG` using comma as a separator -- a vector attribute - -This vector index attribute is broken down as follows: - -- `$.vector_embedding as vector` - `vector` is an alias for `$.vector_embedding` -- `VECTOR` is the type of attribute -- `HSNW` and its six arguments defined the algorithm that will be used, Hierarchical Navigable Small World. - - The vector `TYPE` is `FLOAT32`; each element of the given vector is a 32-bit, floating point number. - - `DIM 3` means that each vector is three dimensional. - - The `DISTANCE_METRIC` is defined as `COSINE`. Other possible values are `IP` and `L2`. - -See the [vector reference](https://redis.io/docs/interact/search-and-query/advanced-concepts/vectors/) for more detailed information about each available option. - -```redis Load some data -JSON.SET user:1 $ '{"user": "samuel", "descr": "Samuel likes mountain and kid\'s bikes.", "labels": "mountain, kids", "vector_embedding": [0.9, 0.7, 0.2]}' -JSON.SET user:2 $ '{"user": "david", "descr": "David likes mountain and kid\'s bikes.", "labels": "mountain, kids", "vector_embedding": [0.7, 0.9, 0.1]}' -JSON.SET user:3 $ '{"user": "pieter", "descr": "Pieter likes kid\'s bikes.", "labels": "kids", "vector_embedding": [0.3, 0.9, 0.2]}' -JSON.SET user:4 $ '{"user": "morti", "descr": "Morti is into kid\'s and road bikes.", "labels": "kids, road", "vector_embedding": [0.1, 0.9, 0.7]}' -``` - -Suppose you want to search for users with preferences similar to the vector `[0.9, 0.7, 0.2]`. Before searching, you'll need to convert the given vector, the JSON array `[0.9, 0.7, 0.2]`, into a flat sequence of bytes. Python's NumPy library can do this: - -```python -import numpy as np - -vector_embedding = [0.9, 0.7, 0.2] -input_vector = np.array(data['vector_embedding']).astype(np.float32).tobytes() -print(input_vector) -b'fff?333?\xcd\xccL>' -``` - -Now you can perform your search using the byte string you obtained from Python as the third to last parameter. - -```redis Search -FT.SEARCH idx:user_prefs "(*)=>[KNN 2 @vector $input_vector]" PARAMS 2 input_vector "fff?333?\xcd\xccL>" DIALECT 2 -``` From 73f2a4b9e32252e2e5ec0b52fc1b0229ff37341c Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 15:48:01 +0200 Subject: [PATCH 15/45] Rename vectors-adv-hash.md to vector_bikes.md --- src/vss/{vectors-adv-hash.md => vector_bikes.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename src/vss/{vectors-adv-hash.md => vector_bikes.md} (100%) diff --git a/src/vss/vectors-adv-hash.md b/src/vss/vector_bikes.md similarity index 100% rename from src/vss/vectors-adv-hash.md rename to src/vss/vector_bikes.md From 93c31592cca8ae84b22c3b34219de6646d447240 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 15:55:41 +0200 Subject: [PATCH 16/45] Update manifest.json --- src/manifest.json | 43 +++++++++++++++++-------------------------- 1 file changed, 17 insertions(+), 26 deletions(-) diff --git a/src/manifest.json b/src/manifest.json index 16347fa..dad3280 100644 --- a/src/manifest.json +++ b/src/manifest.json @@ -34,26 +34,6 @@ "initialIsOpen": true, "path": "/uc/rag.md" } - }, - { - "type": "internal-link", - "id": "ai_assistant", - "label": "Creating an AI Assistant", - "summary": "An assistant that temporarily remembers conversations and understands their meaning.", - "args": { - "initialIsOpen": true, - "path": "/uc/ai_assistant.md" - } - }, - { - "type": "internal-link", - "id": "redis_use_cases_rag", - "label": "Building personalized recommendations", - "summary": "Not just keyword matching, but semantic understanding.", - "args": { - "initialIsOpen": true, - "path": "/uc/personalized_recommendations.md" - } } ] }, @@ -423,18 +403,29 @@ }, { "type": "internal-link", - "id": "vss-vectors-basic", - "label": "Vector search (basic)", + "id": "e-commerce-discovery", + "label": "E-commerce Discovery", + "summary": "Find products by meaning, not just keywords.", "args": { - "path": "/vss/vectors-basic.md" + "path": "/vss/e-commerce-discovery.md" } }, { "type": "internal-link", - "id": "vss-vectors-adv-hash", - "label": "Vector search (advanced)", + "id": "ai_assistant", + "label": "Creating an AI Assistant", + "summary": "An assistant that temporarily remembers conversations and understands their meaning.", "args": { - "path": "/vss/vectors-adv-hash.md" + "path": "/uc/ai_assistant.md" + } + }, + { + "type": "internal-link", + "id": "personalized_recommendations", + "label": "Building personalized recommendations", + "summary": "Suggest movies based on the true meaning of plots or themes.", + "args": { + "path": "/uc/personalized_recommendations.md" } }, { From 936a5b72c9b787a364afa11e18a5dcee2f2fb13f Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 15:55:58 +0200 Subject: [PATCH 17/45] Rename vector_bikes.md to e-commerce-discovery.md --- src/vss/{vector_bikes.md => e-commerce-discovery.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename src/vss/{vector_bikes.md => e-commerce-discovery.md} (100%) diff --git a/src/vss/vector_bikes.md b/src/vss/e-commerce-discovery.md similarity index 100% rename from src/vss/vector_bikes.md rename to src/vss/e-commerce-discovery.md From 8366183fa97951d41b8525ccc39416d36437563c Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:04:56 +0200 Subject: [PATCH 18/45] Create personalized_recommendations.md --- src/vss/personalized_recommendations.md | 202 ++++++++++++++++++++++++ 1 file changed, 202 insertions(+) create mode 100644 src/vss/personalized_recommendations.md diff --git a/src/vss/personalized_recommendations.md b/src/vss/personalized_recommendations.md new file mode 100644 index 0000000..c0f8442 --- /dev/null +++ b/src/vss/personalized_recommendations.md @@ -0,0 +1,202 @@ +This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. + +**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. + +### Architecture Overview +| Layer | Description | Data type | +| ---------- | ---------- | ---------- | +| `Session History`| `Recent conversation context` | List | +| `Rate Limiting` | `Per-user request throttling` | Hash | +| `User Memory` | `Long-term facts and preferences` | Hash | + +### Session History +AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. + +```redis:[run_confirmation=true] Store conversation history +// Add user message to session +LPUSH user:alice:history:session_001 '{"type": "human", "content": "What is the weather like?", "timestamp": 1717935001}' + +// Add AI response +LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It is sunny with 75°F temperature.", "timestamp": 1717935002}' + +// Add another user message +LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bring an umbrella?", "timestamp": 1717935003}' + +// Add AI response +LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' +``` +### Reading Conversation History +You can retrieve recent messages to provide context to the AI. + +```redis:[run_confirmation=no] Read conversation history +// Get the 5 most recent messages +LRANGE user:alice:history:session_001 0 4 + +// Get the full session +LRANGE user:alice:history:session_001 0 -1 +``` +You may want to limit the size of history to retain only the N most recent items. Use LTRIM: +```redis:[run_confirmation=yes] Read conversation history +LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items +``` + +### Session Expiration +Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. + +```redis:[run_confirmation=true] Session expiration +// Set session to expire in 24 hours +EXPIRE user:alice:history:session_001 86400 + +// Set session to expire in 1 hour +EXPIRE user:alice:history:session_001 3600 + +// Check remaining TTL +TTL user:alice:history:session_001 + +// Remove expiration (make persistent) +PERSIST user:alice:history:session_001 +``` + +### Rate Limiting +Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. + +```redis:[run_confirmation=true] Initialize Rate Limiting +// On first request - set counter with 1-minute TTL +HSETEX user:alice:rate_limit EX 60 FIELDS 1 requests_per_minute 1 +``` + +The `HINCR` command allows you to atomically increment the counter, preventing race conditions in high-concurrency scenarios. + +```redis:[run_confirmation=true] Increment Requests +// Increment request counter +HINCRBY user:alice:rate_limit requests_per_minute 1 + +// Check if field exists and get count +HEXISTS user:alice:rate_limit requests_per_minute +HGET user:alice:rate_limit requests_per_minute + +// Check TTL on the hash +TTL user:alice:rate_limit +``` +**Optionally**: if the count exceeds the allowed threshold, deny the operation. + +Different time windows serve different purposes - per-minute prevents burst attacks, per-hour prevents sustained abuse, per-day enforces usage quotas. +```redis:[run_confirmation=true] Rate Limiting with Different Time Windows +// Set multiple rate limits with different TTLs +HSETEX user:alice:rate_limit EX 60 FIELDS 2 requests_per_minute 1 requests_per_hour 1 + +// Daily rate limit (24 hours) +HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 + +// Check all rate limits +HGETALL user:alice:rate_limit +``` + +### User Memory (Persistent Preferences) +AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. + +```redis:[run_confirmation=true] Store User Preferences +// Always secure sensitive data using encryption at rest, access control (Redis ACLs), and comply with data protection laws (e.g., GDPR). +// These values are stored with embeddings to support semantic recall later using vector search. +HSET user:alice:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" + +// Storing communication preference +HSET user:alice:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + +// Store work schedule preference +HSET user:alice:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + +// Store health information +HSET user:alice:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + +// Store pet information +HSET user:alice:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + +// Store family information +HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" +``` + +### Vector Search: Semantic Memory Recall +Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." + +Indexing persistent memory (User Memory) for semantically meaningful search. + +```redis:[run_confirmation=true] Create a Vector Index +// Create index for semantic search +FT.CREATE idx:preferences + ON HASH + PREFIX 1 user: + SCHEMA + user_id TAG + content TEXT + importance NUMERIC + timestamp NUMERIC + embedding VECTOR HNSW 6 + TYPE FLOAT32 + DIM 8 // DIM 8 is only for demonstration purposes. Real embeddings are typically 128–1536 dimensions depending on the model (e.g., sentence-transformers). + DISTANCE_METRIC COSINE +``` + +### Search for most relevant memory entries + +```redis:[run_confirmation=false] Find Top 5 Most Relevant User Memory Items +FT.SEARCH idx:preferences + "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC + DIALECT 2 +``` + +```redis:[run_confirmation=false] Search High-Importance Items Only +FT.SEARCH idx:preferences + "(@user_id:{alice} @importance:[8 +inf]) => [KNN 3 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC + DIALECT 2 +``` +```redis:[run_confirmation=false] Search Recent User Memories +FT.SEARCH idx:preferences + "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + RETURN 3 content timestamp score + SORTBY score ASC + DIALECT 2 +``` + +### Memory State Monitoring +Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. +```redis:[run_confirmation=false] Monitor user sessions +// Get approximate memory usage of session +MEMORY USAGE user:alice:history:session_001 + +// Get session statistics +LLEN user:alice:history:session_001 +TTL user:alice:history:session_001 +``` +### Data Cleanup +Remove all data related to a user (e.g., for GDPR compliance). + +```redis:[run_confirmation=true] Delete user data +// Remove all user data (GDPR compliance) +DEL user:alice:history:session_001 +DEL user:alice:history:session_002 +DEL user:alice:rate_limit +DEL user:alice:pref:001 +DEL user:alice:pref:002 +DEL user:alice:pref:003 +DEL user:alice:personal:001 +DEL user:alice:personal:002 +DEL user:alice:personal:003 +DEL user:alice:work:001 +DEL user:alice:work:002 +DEL user:alice:work:003 +``` + +### Next Steps +Now that your assistant has memory and meaning, you can: + - Combine with RAG Pipelines + - Use sentence-transformers to generate high-dimensional vectors + - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence + - Use Redis ACLs to isolate users, enforce quotas, and monitor usage From 2662848faf4fbb39b38de7991f3651c6db8448ea Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:05:32 +0200 Subject: [PATCH 19/45] Create ai_assistant.md --- src/vss/ai_assistant.md | 202 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 202 insertions(+) create mode 100644 src/vss/ai_assistant.md diff --git a/src/vss/ai_assistant.md b/src/vss/ai_assistant.md new file mode 100644 index 0000000..c0f8442 --- /dev/null +++ b/src/vss/ai_assistant.md @@ -0,0 +1,202 @@ +This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. + +**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. + +### Architecture Overview +| Layer | Description | Data type | +| ---------- | ---------- | ---------- | +| `Session History`| `Recent conversation context` | List | +| `Rate Limiting` | `Per-user request throttling` | Hash | +| `User Memory` | `Long-term facts and preferences` | Hash | + +### Session History +AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. + +```redis:[run_confirmation=true] Store conversation history +// Add user message to session +LPUSH user:alice:history:session_001 '{"type": "human", "content": "What is the weather like?", "timestamp": 1717935001}' + +// Add AI response +LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It is sunny with 75°F temperature.", "timestamp": 1717935002}' + +// Add another user message +LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bring an umbrella?", "timestamp": 1717935003}' + +// Add AI response +LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' +``` +### Reading Conversation History +You can retrieve recent messages to provide context to the AI. + +```redis:[run_confirmation=no] Read conversation history +// Get the 5 most recent messages +LRANGE user:alice:history:session_001 0 4 + +// Get the full session +LRANGE user:alice:history:session_001 0 -1 +``` +You may want to limit the size of history to retain only the N most recent items. Use LTRIM: +```redis:[run_confirmation=yes] Read conversation history +LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items +``` + +### Session Expiration +Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. + +```redis:[run_confirmation=true] Session expiration +// Set session to expire in 24 hours +EXPIRE user:alice:history:session_001 86400 + +// Set session to expire in 1 hour +EXPIRE user:alice:history:session_001 3600 + +// Check remaining TTL +TTL user:alice:history:session_001 + +// Remove expiration (make persistent) +PERSIST user:alice:history:session_001 +``` + +### Rate Limiting +Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. + +```redis:[run_confirmation=true] Initialize Rate Limiting +// On first request - set counter with 1-minute TTL +HSETEX user:alice:rate_limit EX 60 FIELDS 1 requests_per_minute 1 +``` + +The `HINCR` command allows you to atomically increment the counter, preventing race conditions in high-concurrency scenarios. + +```redis:[run_confirmation=true] Increment Requests +// Increment request counter +HINCRBY user:alice:rate_limit requests_per_minute 1 + +// Check if field exists and get count +HEXISTS user:alice:rate_limit requests_per_minute +HGET user:alice:rate_limit requests_per_minute + +// Check TTL on the hash +TTL user:alice:rate_limit +``` +**Optionally**: if the count exceeds the allowed threshold, deny the operation. + +Different time windows serve different purposes - per-minute prevents burst attacks, per-hour prevents sustained abuse, per-day enforces usage quotas. +```redis:[run_confirmation=true] Rate Limiting with Different Time Windows +// Set multiple rate limits with different TTLs +HSETEX user:alice:rate_limit EX 60 FIELDS 2 requests_per_minute 1 requests_per_hour 1 + +// Daily rate limit (24 hours) +HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 + +// Check all rate limits +HGETALL user:alice:rate_limit +``` + +### User Memory (Persistent Preferences) +AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. + +```redis:[run_confirmation=true] Store User Preferences +// Always secure sensitive data using encryption at rest, access control (Redis ACLs), and comply with data protection laws (e.g., GDPR). +// These values are stored with embeddings to support semantic recall later using vector search. +HSET user:alice:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" + +// Storing communication preference +HSET user:alice:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + +// Store work schedule preference +HSET user:alice:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + +// Store health information +HSET user:alice:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + +// Store pet information +HSET user:alice:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + +// Store family information +HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" +``` + +### Vector Search: Semantic Memory Recall +Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." + +Indexing persistent memory (User Memory) for semantically meaningful search. + +```redis:[run_confirmation=true] Create a Vector Index +// Create index for semantic search +FT.CREATE idx:preferences + ON HASH + PREFIX 1 user: + SCHEMA + user_id TAG + content TEXT + importance NUMERIC + timestamp NUMERIC + embedding VECTOR HNSW 6 + TYPE FLOAT32 + DIM 8 // DIM 8 is only for demonstration purposes. Real embeddings are typically 128–1536 dimensions depending on the model (e.g., sentence-transformers). + DISTANCE_METRIC COSINE +``` + +### Search for most relevant memory entries + +```redis:[run_confirmation=false] Find Top 5 Most Relevant User Memory Items +FT.SEARCH idx:preferences + "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC + DIALECT 2 +``` + +```redis:[run_confirmation=false] Search High-Importance Items Only +FT.SEARCH idx:preferences + "(@user_id:{alice} @importance:[8 +inf]) => [KNN 3 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC + DIALECT 2 +``` +```redis:[run_confirmation=false] Search Recent User Memories +FT.SEARCH idx:preferences + "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + RETURN 3 content timestamp score + SORTBY score ASC + DIALECT 2 +``` + +### Memory State Monitoring +Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. +```redis:[run_confirmation=false] Monitor user sessions +// Get approximate memory usage of session +MEMORY USAGE user:alice:history:session_001 + +// Get session statistics +LLEN user:alice:history:session_001 +TTL user:alice:history:session_001 +``` +### Data Cleanup +Remove all data related to a user (e.g., for GDPR compliance). + +```redis:[run_confirmation=true] Delete user data +// Remove all user data (GDPR compliance) +DEL user:alice:history:session_001 +DEL user:alice:history:session_002 +DEL user:alice:rate_limit +DEL user:alice:pref:001 +DEL user:alice:pref:002 +DEL user:alice:pref:003 +DEL user:alice:personal:001 +DEL user:alice:personal:002 +DEL user:alice:personal:003 +DEL user:alice:work:001 +DEL user:alice:work:002 +DEL user:alice:work:003 +``` + +### Next Steps +Now that your assistant has memory and meaning, you can: + - Combine with RAG Pipelines + - Use sentence-transformers to generate high-dimensional vectors + - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence + - Use Redis ACLs to isolate users, enforce quotas, and monitor usage From 9e63170d06101bcd8ee7ac9cad2f90d271aa25eb Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:09:39 +0200 Subject: [PATCH 20/45] Update manifest.json --- src/manifest.json | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/manifest.json b/src/manifest.json index dad3280..e78ad52 100644 --- a/src/manifest.json +++ b/src/manifest.json @@ -412,20 +412,20 @@ }, { "type": "internal-link", - "id": "ai_assistant", - "label": "Creating an AI Assistant", - "summary": "An assistant that temporarily remembers conversations and understands their meaning.", + "id": "personalized_recommendations", + "label": "Building personalized recommendations", + "summary": "Suggest movies based on the true meaning of plots or themes.", "args": { - "path": "/uc/ai_assistant.md" + "path": "/uc/personalized_recommendations.md" } }, { "type": "internal-link", - "id": "personalized_recommendations", - "label": "Building personalized recommendations", - "summary": "Suggest movies based on the true meaning of plots or themes.", + "id": "ai_assistant", + "label": "Creating an AI Assistant", + "summary": "An assistant that temporarily remembers conversations and understands their meaning.", "args": { - "path": "/uc/personalized_recommendations.md" + "path": "/uc/ai_assistant.md" } }, { From 57635eec0e4f415bfcc63db5dcecba516a948da1 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:15:37 +0200 Subject: [PATCH 21/45] Update hashes.md --- src/ds/hashes.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/ds/hashes.md b/src/ds/hashes.md index 0342214..ae3ba45 100644 --- a/src/ds/hashes.md +++ b/src/ds/hashes.md @@ -44,7 +44,7 @@ HINCRBY bike:1 price 100 HINCRBY bike:1 price -100 ``` -New in [Redis Community Edition version 7.4](https://hub.docker.com/layers/redis/redis-stack/7.4.0-v0/images/sha256-3e3c86603a81712d1311bc619ad124de15b2dca2b50722f23a4502b4d4054ba8) is the ability set the expiration time or the remaining time-to-live (TTL) for individual hash fields. This is called hash field expiration (HFE). HFE works just like [key expiration](https://redis.io/docs/latest/develop/use/keyspace/?utm_source=redisinsight&utm_medium=main&utm_campaign=tutorials#key-expiration) and includes the following commands: +Available in [Redis 8](https://hub.docker.com/layers/library/redis/8.0.3/images/sha256-426e6823fb1778e8c49f327f9e5af00e505a7fca726ffe11b7930eb1d99ef5fd) is the ability set the expiration time or the remaining time-to-live (TTL) for individual hash fields. This is called hash field expiration (HFE). HFE works just like [key expiration](https://redis.io/docs/latest/develop/use/keyspace/?utm_source=redisinsight&utm_medium=main&utm_campaign=tutorials#key-expiration) and includes the following commands: - `hexpire` - set an expiration (time-to-live or TTL) in seconds on a hash key's field(s). - `hexpireat` - set a TTL as an absolute Unix timestamp (seconds since Unix epoch) on a hash key's field(s). @@ -80,6 +80,7 @@ HGETALL hash ``` ### Resources +- A tutorial on building an [AI assistant](redisinsight:_?tutorialId=ai_assistant) using hash field expiration. - Hash type [reference page](https://redis.io/docs/data-types/hashes?utm_source=redisinsight&utm_medium=main&utm_campaign=tutorials). - Entire set of [Redis hash commands](https://redis.io/commands/?group=hash?utm_source=redisinsight&utm_medium=main&utm_campaign=tutorials). - Check out [Get started with Redis](https://university.redis.io/learningpath/14q8m6gilfwltm?utm_source=redisinsight&utm_medium=main&utm_campaign=tutorials) learning path on Redis University for an introduction to working with all core data structures in Redis. From 7aad062abd2f16f4cfb631420188dab08bffcf1f Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:17:39 +0200 Subject: [PATCH 22/45] Mentioning Redis 8 instead of Redis Stack --- src/ds/ts/intro.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ds/ts/intro.md b/src/ds/ts/intro.md index 464ea32..8b72dc9 100644 --- a/src/ds/ts/intro.md +++ b/src/ds/ts/intro.md @@ -16,8 +16,8 @@ You can ingest and query millions of samples and events at the speed of Redis. ### Prerequisites -[Redis Stack Server](https://redis.io/downloads/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial) >=7.2.0-v7 \ +[Redis 8](https://hub.docker.com/layers/library/redis/8.0.3/images/sha256-426e6823fb1778e8c49f327f9e5af00e505a7fca726ffe11b7930eb1d99ef5fd) or higher \ OR \ [RedisTimeSeries](https://oss.redis.com/redistimeseries/) >=1.10.11 \ OR \ -A free Redis Stack instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). From 6e50a31a8e4b2bdcbfaad3936b5d25297b8dbc75 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:18:43 +0200 Subject: [PATCH 23/45] Mentioning Redis 8 instead of Redis Stack --- src/ds/prob/intro.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ds/prob/intro.md b/src/ds/prob/intro.md index 96f6ace..6b9293b 100644 --- a/src/ds/prob/intro.md +++ b/src/ds/prob/intro.md @@ -48,8 +48,8 @@ The following data structures trade perfect accuracy for extreme memory efficien ### Prerequisites -[Redis Stack Server](https://redis.io/downloads/?utm_source=redisinsight&utm_medium=app&utm_campaign=probabilistic_tutorial) >=7.2.0-v7 \ +[Redis 8](https://hub.docker.com/layers/library/redis/8.0.3/images/sha256-426e6823fb1778e8c49f327f9e5af00e505a7fca726ffe11b7930eb1d99ef5fd) or higher \ OR \ [RedisBloom](https://oss.redis.com/redisbloom/) >=2.6.10 \ OR \ -A free Redis Stack instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=probabilistic_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). From c920ba01dac4efa52072cb30208b97ae9447879b Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:20:08 +0200 Subject: [PATCH 24/45] Mentioning JSON data structure instead of Redis Stack --- src/ds/json/adv-jsonpath.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ds/json/adv-jsonpath.md b/src/ds/json/adv-jsonpath.md index c321c7c..92e4be9 100644 --- a/src/ds/json/adv-jsonpath.md +++ b/src/ds/json/adv-jsonpath.md @@ -1,5 +1,5 @@ [JSONPath](https://goessner.net/articles/JsonPath/) expressions help you access specific elements within a JSON document, which is similar to how XPATH works for XML documents. -JSONPath support was added to Redis Stack in version 2.0. +JSONPath support was added to the JSON data structure in version 2.0. Before that, [a legacy form of pathing](https://redis.io/docs/data-types/json/path/#legacy-path-syntax) was supported. Only JSONPath will be discussed in this tutorial. @@ -133,7 +133,7 @@ JSON.GET obj2 $.b[0] JSON.GET obj2 $.*[0] ``` -Redis Stack also supports slice syntax for arrays: `[start:`end`:`step`]`, where `start`, `end`, and `step` are indexes. +JSON data structure also supports slice syntax for arrays: `[start:`end`:`step`]`, where `start`, `end`, and `step` are indexes. If the current node is an array, an array containing elements extracted from an array are returned, based on a `start` index, an `end` index, and a `step` index. Array indexes are zero-based; the first element is index 0. Start Index is inclusive; End index is not inclusive. The following rules apply: From 0b1ee92680a1f76ba5252f2f435da68e18de0b21 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:20:40 +0200 Subject: [PATCH 25/45] Mentioning Redis 8 instead of Redis Stack --- src/ds/json/intro.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ds/json/intro.md b/src/ds/json/intro.md index a6829b9..d99d741 100644 --- a/src/ds/json/intro.md +++ b/src/ds/json/intro.md @@ -10,8 +10,8 @@ Primary features include: ### Prerequisites -[Redis Stack](https://redis.io/downloads/?utm_source=redisinsight&utm_medium=app&utm_campaign=json_tutorial) >=7.2.0-v7 \ +[Redis 8](https://hub.docker.com/layers/library/redis/8.0.3/images/sha256-426e6823fb1778e8c49f327f9e5af00e505a7fca726ffe11b7930eb1d99ef5fd) or higher \ OR \ [RedisJSON](https://github.com/RedisJSON/RedisJSON/) >=2.6.8 \ OR \ -A free Redis Stack instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=json_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). From 16f8cc032d8d2b7a2f1edc7787d0cbdebbcf37f8 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:21:17 +0200 Subject: [PATCH 26/45] Update more-adv-jsonpath.md --- src/ds/json/more-adv-jsonpath.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ds/json/more-adv-jsonpath.md b/src/ds/json/more-adv-jsonpath.md index 73739d5..b47e4b7 100644 --- a/src/ds/json/more-adv-jsonpath.md +++ b/src/ds/json/more-adv-jsonpath.md @@ -112,7 +112,7 @@ JSON.GET obj2 '$.a[?(@!="a")]' // [1,2,3,"b","c",false,true,["a",1],{"a":1},{"b" #### Relational use cases using regular expression with the `=~` operator **Note**: -> Redis Stack uses [Rust regular expressions syntax](https://docs.rs/regex/latest/regex/#syntax). Invalid regular expressions are not evaluated. +> JSON data structure uses [Rust regular expressions syntax](https://docs.rs/regex/latest/regex/#syntax). Invalid regular expressions are not evaluated. There are two cases: From 7a557419c7a8b4d177cf1ed027d5251e95c7fdbf Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:26:50 +0200 Subject: [PATCH 27/45] Mentioning RQE and Redis 8 instead of Redis Stack --- src/sq/geospatial.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/sq/geospatial.md b/src/sq/geospatial.md index 5f63394..4fdc998 100644 --- a/src/sq/geospatial.md +++ b/src/sq/geospatial.md @@ -1,4 +1,4 @@ -Redis Stack's geospatial feature allows you to query for data associated with geographic locations. You can either query for locations within a specific radius or based on geometric shapes, such as polygons. A polygon shape could, for instance, represent a lake or the layout of a building. +Redis Query Engine's geospatial feature allows you to query for data associated with geographic locations. You can either query for locations within a specific radius or based on geometric shapes, such as polygons. A polygon shape could, for instance, represent a lake or the layout of a building. The examples in this article use the following schema: @@ -9,7 +9,7 @@ The examples in this article use the following schema: **Note**: -> Redis Stack version 7.2.0 or higher is required to use the `GEOSHAPE` field type. +> We recommend using Redis 8 or higher to use the `GEOSHAPE` field type. ```redis:[run_confirmation=true] Create the bike shop idx:bicycle FT.CREATE From 884b9ad5b1bfaac4958e134e3833ae8aa9b8cbef Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:36:54 +0200 Subject: [PATCH 28/45] Update intro.md --- src/sq/intro.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/sq/intro.md b/src/sq/intro.md index dde922b..2113fdb 100644 --- a/src/sq/intro.md +++ b/src/sq/intro.md @@ -1,4 +1,4 @@ -Redis Stack offers an enhanced Redis experience via the following Redis Query Engine features: +The Redis Query Engine offers an enhanced Redis experience via the following search and query features: - A rich query language - Incremental indexing on JSON and hash documents @@ -7,7 +7,7 @@ Redis Stack offers an enhanced Redis experience via the following Redis Query En - Geospatial queries - Aggregations -The Redis Query Engine features of Redis Stack allow you to use Redis as a: +The Redis Query Engine features you to use Redis as a: - Document database - Vector database @@ -16,8 +16,8 @@ The Redis Query Engine features of Redis Stack allow you to use Redis as a: ### Prerequisites -[Redis Stack](https://redis.io/downloads/?utm_source=redisinsight&utm_medium=app&utm_campaign=query_engine_tutorial) >=7.4.0-v0 \ +[Redis 8](https://hub.docker.com/layers/library/redis/8.0.3/images/sha256-426e6823fb1778e8c49f327f9e5af00e505a7fca726ffe11b7930eb1d99ef5fd) or higher \ OR \ [Redis Query Engine](https://github.com/RediSearch/RediSearch/) >=2.10.5 \ OR \ -A free Redis Stack instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=query_engine_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). From 0a9e338af4cd1455e2a6affae6028c3fd0a52205 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:38:11 +0200 Subject: [PATCH 29/45] Update intro.md --- src/ds/json/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ds/json/intro.md b/src/ds/json/intro.md index d99d741..4132230 100644 --- a/src/ds/json/intro.md +++ b/src/ds/json/intro.md @@ -1,4 +1,4 @@ -The JSON capability of Redis Stack provides JavaScript Object Notation (JSON) support for Redis, which allows Redis to function as a document database. +The JSON capability provides JavaScript Object Notation (JSON) support for Redis, which allows Redis to function as a document database. It lets you store, update, and retrieve JSON values in a Redis database, similar to any other Redis data type. Redis JSON also works seamlessly with Search and Query to let you index and query JSON documents. Primary features include: From 87294f59caddf5bec6924b17ad6a55dbc375cb8c Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 16:39:15 +0200 Subject: [PATCH 30/45] Mentioning Redis 8 instead of Redis Stack --- src/vss/intro.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/vss/intro.md b/src/vss/intro.md index 182476c..ce09de7 100644 --- a/src/vss/intro.md +++ b/src/vss/intro.md @@ -16,7 +16,7 @@ Machine learning models have facilitated the rise of embeddings as a widely embr Given a suitable machine learning model, the generated embeddings can encapsulate complex patterns and semantic meanings inherent in data. -You can use Redis Stack as a vector database, which allows you to: +You can use Redis 8 as a vector database, which allows you to: - Store vectors and the associated metadata within hashes or JSON documents - Retrieve vectors @@ -25,8 +25,8 @@ You can use Redis Stack as a vector database, which allows you to: ### Prerequisites -[Redis Stack](https://redis.io/downloads/?utm_source=redisinsight&utm_medium=app&utm_campaign=vss_tutorial) >=7.2.0-v7 \ +[Redis 8](https://hub.docker.com/layers/library/redis/8.0.3/images/sha256-426e6823fb1778e8c49f327f9e5af00e505a7fca726ffe11b7930eb1d99ef5fd) or higher \ OR \ [RediSearch](https://github.com/RediSearch/RediSearch/) >=2.8.11 \ OR \ -A free Redis Stack instance on [Redis Cloud](https://redis.com/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=vss_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). From 9e391cca1537c31900b64ea230f8724e8dfe2e1d Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:00:25 +0200 Subject: [PATCH 31/45] Update manifest.json --- src/manifest.json | 258 +++++++++++++++++++++++----------------------- 1 file changed, 129 insertions(+), 129 deletions(-) diff --git a/src/manifest.json b/src/manifest.json index e78ad52..7f72038 100644 --- a/src/manifest.json +++ b/src/manifest.json @@ -37,6 +37,135 @@ } ] }, + { + "type": "group", + "id": "vss", + "label": "Vector search explained", + "args": { + "initialIsOpen": false + }, + "children": [ + { + "type": "internal-link", + "id": "vss-intro", + "label": "Introduction", + "summary": "Understand how to use Redis as a vector database.", + "args": { + "path": "/vss/intro.md" + } + }, + { + "type": "internal-link", + "id": "e-commerce-discovery", + "label": "E-commerce Discovery", + "summary": "Find products by meaning, not just keywords.", + "args": { + "path": "/vss/e-commerce-discovery.md" + } + }, + { + "type": "internal-link", + "id": "personalized_recommendations", + "label": "Building personalized recommendations", + "summary": "Suggest movies based on the true meaning of plots or themes.", + "args": { + "path": "/uc/personalized_recommendations.md" + } + }, + { + "type": "internal-link", + "id": "ai_assistant", + "label": "Creating an AI Assistant", + "summary": "An assistant that temporarily remembers conversations and understands their meaning.", + "args": { + "path": "/uc/ai_assistant.md" + } + }, + { + "type": "internal-link", + "id": "vss-learn-more", + "label": "Learn more", + "args": { + "path": "/vss/learn-more.md" + } + } + ] + }, + { + "type": "group", + "id": "sq", + "label": "How to query your data", + "args": { + "initialIsOpen": false + }, + "children": [ + { + "type": "internal-link", + "id": "sq-intro", + "label": "Introduction", + "summary": "Try real-time searching and perform complex structured queries without compromising the database performance.", + "args": { + "path": "/sq/intro.md" + } + }, + { + "type": "internal-link", + "id": "sq-exact-match", + "label": "Exact match", + "args": { + "path": "/sq/exact-match.md" + } + }, + { + "type": "internal-link", + "id": "sq-full-text", + "label": "Full-text search", + "args": { + "path": "/sq/full-text.md" + } + }, + { + "type": "internal-link", + "id": "sq-range", + "label": "Range queries", + "args": { + "path": "/sq/range.md" + } + }, + { + "type": "internal-link", + "id": "sq-geospatial", + "label": "Geospatial queries", + "args": { + "path": "/sq/geospatial.md" + } + }, + { + "type": "internal-link", + "id": "sq-combined", + "label": "Combined queries", + "args": { + "path": "/sq/combined.md" + } + }, + { + "type": "internal-link", + "id": "sq-aggregations", + "label": "Analytic and transformative queries", + "args": { + "path": "/sq/aggregations.md" + } + }, + { + "type": "internal-link", + "id": "sq-learn-more", + "label": "Learn more", + "args": { + "path": "/sq/learn-more.md" + } + } + ] + }, { "type": "group", "id": "ds", @@ -308,135 +437,6 @@ ] } ] - }, - { - "type": "group", - "id": "sq", - "label": "How to query your data", - "args": { - "initialIsOpen": false - }, - "children": [ - { - "type": "internal-link", - "id": "sq-intro", - "label": "Introduction", - "summary": "Try real-time searching and perform complex structured queries without compromising the database performance.", - "args": { - "path": "/sq/intro.md" - } - }, - { - "type": "internal-link", - "id": "sq-exact-match", - "label": "Exact match", - "args": { - "path": "/sq/exact-match.md" - } - }, - { - "type": "internal-link", - "id": "sq-full-text", - "label": "Full-text search", - "args": { - "path": "/sq/full-text.md" - } - }, - { - "type": "internal-link", - "id": "sq-range", - "label": "Range queries", - "args": { - "path": "/sq/range.md" - } - }, - { - "type": "internal-link", - "id": "sq-geospatial", - "label": "Geospatial queries", - "args": { - "path": "/sq/geospatial.md" - } - }, - { - "type": "internal-link", - "id": "sq-combined", - "label": "Combined queries", - "args": { - "path": "/sq/combined.md" - } - }, - { - "type": "internal-link", - "id": "sq-aggregations", - "label": "Analytic and transformative queries", - "args": { - "path": "/sq/aggregations.md" - } - }, - { - "type": "internal-link", - "id": "sq-learn-more", - "label": "Learn more", - "args": { - "path": "/sq/learn-more.md" - } - } - ] - }, - { - "type": "group", - "id": "vss", - "label": "Vector search explained", - "args": { - "initialIsOpen": false - }, - "children": [ - { - "type": "internal-link", - "id": "vss-intro", - "label": "Introduction", - "summary": "Understand how to use Redis as a vector database.", - "args": { - "path": "/vss/intro.md" - } - }, - { - "type": "internal-link", - "id": "e-commerce-discovery", - "label": "E-commerce Discovery", - "summary": "Find products by meaning, not just keywords.", - "args": { - "path": "/vss/e-commerce-discovery.md" - } - }, - { - "type": "internal-link", - "id": "personalized_recommendations", - "label": "Building personalized recommendations", - "summary": "Suggest movies based on the true meaning of plots or themes.", - "args": { - "path": "/uc/personalized_recommendations.md" - } - }, - { - "type": "internal-link", - "id": "ai_assistant", - "label": "Creating an AI Assistant", - "summary": "An assistant that temporarily remembers conversations and understands their meaning.", - "args": { - "path": "/uc/ai_assistant.md" - } - }, - { - "type": "internal-link", - "id": "vss-learn-more", - "label": "Learn more", - "args": { - "path": "/vss/learn-more.md" - } - } - ] } ] } From 9fee51965d3d407dcccea883d55cd8960a343b56 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:10:51 +0200 Subject: [PATCH 32/45] Update intro.md --- src/ds/json/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ds/json/intro.md b/src/ds/json/intro.md index 4132230..7d1975d 100644 --- a/src/ds/json/intro.md +++ b/src/ds/json/intro.md @@ -14,4 +14,4 @@ Primary features include: OR \ [RedisJSON](https://github.com/RedisJSON/RedisJSON/) >=2.6.8 \ OR \ -A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=json_tutorial). From 261107733e44c8d458e855b5755eac3ee86ed78e Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:11:23 +0200 Subject: [PATCH 33/45] Update intro.md --- src/ds/prob/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ds/prob/intro.md b/src/ds/prob/intro.md index 6b9293b..ce5e930 100644 --- a/src/ds/prob/intro.md +++ b/src/ds/prob/intro.md @@ -52,4 +52,4 @@ The following data structures trade perfect accuracy for extreme memory efficien OR \ [RedisBloom](https://oss.redis.com/redisbloom/) >=2.6.10 \ OR \ -A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=probabilistic_tutorial). From 40b59fac929b6d9af99301b023ff251fdfb9d0c3 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:12:19 +0200 Subject: [PATCH 34/45] Update intro.md --- src/sq/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/sq/intro.md b/src/sq/intro.md index 2113fdb..5aa2658 100644 --- a/src/sq/intro.md +++ b/src/sq/intro.md @@ -20,4 +20,4 @@ The Redis Query Engine features you to use Redis as a: OR \ [Redis Query Engine](https://github.com/RediSearch/RediSearch/) >=2.10.5 \ OR \ -A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=redis_query_engine_tutorial). From 3e463664379d15d8bf3eae56642c6cf5c8f046a3 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:12:58 +0200 Subject: [PATCH 35/45] Update intro.md --- src/vss/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/vss/intro.md b/src/vss/intro.md index ce09de7..dc9020e 100644 --- a/src/vss/intro.md +++ b/src/vss/intro.md @@ -29,4 +29,4 @@ You can use Redis 8 as a vector database, which allows you to: OR \ [RediSearch](https://github.com/RediSearch/RediSearch/) >=2.8.11 \ OR \ -A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=timeseries_tutorial). +A free Redis instance on [Redis Cloud](https://redis.io/try-free/?utm_source=redisinsight&utm_medium=app&utm_campaign=vector_search_tutorial). From 2822d9af5ad17372d302882aa72a3bd1bbb335b0 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:19:36 +0200 Subject: [PATCH 36/45] Update ai_assistant.md --- src/uc/ai_assistant.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index c0f8442..c3e2af2 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -2,14 +2,14 @@ This tutorial demonstrates how to build an AI assistant's memory system with Red **Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. -### Architecture Overview +### Architecture overview | Layer | Description | Data type | | ---------- | ---------- | ---------- | | `Session History`| `Recent conversation context` | List | | `Rate Limiting` | `Per-user request throttling` | Hash | | `User Memory` | `Long-term facts and preferences` | Hash | -### Session History +### Session history AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. ```redis:[run_confirmation=true] Store conversation history @@ -25,7 +25,7 @@ LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bri // Add AI response LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' ``` -### Reading Conversation History +### Reading conversation history You can retrieve recent messages to provide context to the AI. ```redis:[run_confirmation=no] Read conversation history @@ -40,7 +40,7 @@ You may want to limit the size of history to retain only the N most recent items LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items ``` -### Session Expiration +### Session expiration Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. ```redis:[run_confirmation=true] Session expiration @@ -57,7 +57,7 @@ TTL user:alice:history:session_001 PERSIST user:alice:history:session_001 ``` -### Rate Limiting +### Rate limiting Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. ```redis:[run_confirmation=true] Initialize Rate Limiting @@ -92,7 +92,7 @@ HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 HGETALL user:alice:rate_limit ``` -### User Memory (Persistent Preferences) +### User memory (persistent preferences) AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. ```redis:[run_confirmation=true] Store User Preferences @@ -116,7 +116,7 @@ HSET user:alice:personal:002 user_id "alice" content "has a golden retriever nam HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" ``` -### Vector Search: Semantic Memory Recall +### Vector search: semantic memory recall Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." Indexing persistent memory (User Memory) for semantically meaningful search. @@ -165,7 +165,7 @@ FT.SEARCH idx:preferences DIALECT 2 ``` -### Memory State Monitoring +### Memory state monitoring Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. ```redis:[run_confirmation=false] Monitor user sessions // Get approximate memory usage of session @@ -175,7 +175,7 @@ MEMORY USAGE user:alice:history:session_001 LLEN user:alice:history:session_001 TTL user:alice:history:session_001 ``` -### Data Cleanup +### Data cleanup Remove all data related to a user (e.g., for GDPR compliance). ```redis:[run_confirmation=true] Delete user data @@ -194,7 +194,7 @@ DEL user:alice:work:002 DEL user:alice:work:003 ``` -### Next Steps +### Next steps Now that your assistant has memory and meaning, you can: - Combine with RAG Pipelines - Use sentence-transformers to generate high-dimensional vectors From 9d7c2fefd84979181eee2c4ee6d50562cd1c60fd Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:20:35 +0200 Subject: [PATCH 37/45] Apply suggestions from code review Co-authored-by: David Dougherty --- src/ds/json/adv-jsonpath.md | 2 +- src/ds/json/more-adv-jsonpath.md | 2 +- src/manifest.json | 2 +- src/sq/intro.md | 2 +- src/uc/ai_assistant.md | 6 +++--- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/src/ds/json/adv-jsonpath.md b/src/ds/json/adv-jsonpath.md index 92e4be9..6adc470 100644 --- a/src/ds/json/adv-jsonpath.md +++ b/src/ds/json/adv-jsonpath.md @@ -133,7 +133,7 @@ JSON.GET obj2 $.b[0] JSON.GET obj2 $.*[0] ``` -JSON data structure also supports slice syntax for arrays: `[start:`end`:`step`]`, where `start`, `end`, and `step` are indexes. +The JSON data structure also supports slice syntax for arrays: `[start:`end`:`step`]`, where `start`, `end`, and `step` are indexes. If the current node is an array, an array containing elements extracted from an array are returned, based on a `start` index, an `end` index, and a `step` index. Array indexes are zero-based; the first element is index 0. Start Index is inclusive; End index is not inclusive. The following rules apply: diff --git a/src/ds/json/more-adv-jsonpath.md b/src/ds/json/more-adv-jsonpath.md index b47e4b7..6322389 100644 --- a/src/ds/json/more-adv-jsonpath.md +++ b/src/ds/json/more-adv-jsonpath.md @@ -112,7 +112,7 @@ JSON.GET obj2 '$.a[?(@!="a")]' // [1,2,3,"b","c",false,true,["a",1],{"a":1},{"b" #### Relational use cases using regular expression with the `=~` operator **Note**: -> JSON data structure uses [Rust regular expressions syntax](https://docs.rs/regex/latest/regex/#syntax). Invalid regular expressions are not evaluated. +> The JSON data structure uses [Rust regular expressions syntax](https://docs.rs/regex/latest/regex/#syntax). Invalid regular expressions are not evaluated. There are two cases: diff --git a/src/manifest.json b/src/manifest.json index 7f72038..fe0b65c 100644 --- a/src/manifest.json +++ b/src/manifest.json @@ -103,7 +103,7 @@ "type": "internal-link", "id": "sq-intro", "label": "Introduction", - "summary": "Try real-time searching and perform complex structured queries without compromising the database performance.", + "summary": "Try real-time searching and perform complex structured queries without compromising database performance.", "args": { "path": "/sq/intro.md" } diff --git a/src/sq/intro.md b/src/sq/intro.md index 5aa2658..84e5711 100644 --- a/src/sq/intro.md +++ b/src/sq/intro.md @@ -1,4 +1,4 @@ -The Redis Query Engine offers an enhanced Redis experience via the following search and query features: +The Redis Query Engine offers an enhanced Redis experience using the following search and query features: - A rich query language - Incremental indexing on JSON and hash documents diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md index c3e2af2..3ba2a0a 100644 --- a/src/uc/ai_assistant.md +++ b/src/uc/ai_assistant.md @@ -1,6 +1,6 @@ This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. -**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. +**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTLs for hashes - ideal for rate limiting to ensure fair resource usage. ### Architecture overview | Layer | Description | Data type | @@ -58,7 +58,7 @@ PERSIST user:alice:history:session_001 ``` ### Rate limiting -Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. +Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTLs applied using `HSETEX` are ideal for this. ```redis:[run_confirmation=true] Initialize Rate Limiting // On first request - set counter with 1-minute TTL @@ -80,7 +80,7 @@ TTL user:alice:rate_limit ``` **Optionally**: if the count exceeds the allowed threshold, deny the operation. -Different time windows serve different purposes - per-minute prevents burst attacks, per-hour prevents sustained abuse, per-day enforces usage quotas. +Different time windows serve different purposes: per-minute windows prevent burst attacks; per-hour windows prevent sustained abuse; and per-day windows enforce usage quotas. ```redis:[run_confirmation=true] Rate Limiting with Different Time Windows // Set multiple rate limits with different TTLs HSETEX user:alice:rate_limit EX 60 FIELDS 2 requests_per_minute 1 requests_per_hour 1 From 81b9d5cfa9bced1df40663a67180767319891e9a Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:21:56 +0200 Subject: [PATCH 38/45] Update ai_assistant.md --- src/vss/ai_assistant.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/src/vss/ai_assistant.md b/src/vss/ai_assistant.md index c0f8442..c3e2af2 100644 --- a/src/vss/ai_assistant.md +++ b/src/vss/ai_assistant.md @@ -2,14 +2,14 @@ This tutorial demonstrates how to build an AI assistant's memory system with Red **Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. -### Architecture Overview +### Architecture overview | Layer | Description | Data type | | ---------- | ---------- | ---------- | | `Session History`| `Recent conversation context` | List | | `Rate Limiting` | `Per-user request throttling` | Hash | | `User Memory` | `Long-term facts and preferences` | Hash | -### Session History +### Session history AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. ```redis:[run_confirmation=true] Store conversation history @@ -25,7 +25,7 @@ LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bri // Add AI response LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' ``` -### Reading Conversation History +### Reading conversation history You can retrieve recent messages to provide context to the AI. ```redis:[run_confirmation=no] Read conversation history @@ -40,7 +40,7 @@ You may want to limit the size of history to retain only the N most recent items LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items ``` -### Session Expiration +### Session expiration Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. ```redis:[run_confirmation=true] Session expiration @@ -57,7 +57,7 @@ TTL user:alice:history:session_001 PERSIST user:alice:history:session_001 ``` -### Rate Limiting +### Rate limiting Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. ```redis:[run_confirmation=true] Initialize Rate Limiting @@ -92,7 +92,7 @@ HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 HGETALL user:alice:rate_limit ``` -### User Memory (Persistent Preferences) +### User memory (persistent preferences) AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. ```redis:[run_confirmation=true] Store User Preferences @@ -116,7 +116,7 @@ HSET user:alice:personal:002 user_id "alice" content "has a golden retriever nam HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" ``` -### Vector Search: Semantic Memory Recall +### Vector search: semantic memory recall Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." Indexing persistent memory (User Memory) for semantically meaningful search. @@ -165,7 +165,7 @@ FT.SEARCH idx:preferences DIALECT 2 ``` -### Memory State Monitoring +### Memory state monitoring Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. ```redis:[run_confirmation=false] Monitor user sessions // Get approximate memory usage of session @@ -175,7 +175,7 @@ MEMORY USAGE user:alice:history:session_001 LLEN user:alice:history:session_001 TTL user:alice:history:session_001 ``` -### Data Cleanup +### Data cleanup Remove all data related to a user (e.g., for GDPR compliance). ```redis:[run_confirmation=true] Delete user data @@ -194,7 +194,7 @@ DEL user:alice:work:002 DEL user:alice:work:003 ``` -### Next Steps +### Next steps Now that your assistant has memory and meaning, you can: - Combine with RAG Pipelines - Use sentence-transformers to generate high-dimensional vectors From 432a241bd7d547c21dd95f622fcb7136d6c956cb Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:24:08 +0200 Subject: [PATCH 39/45] Delete src/uc/ai_assistant.md --- src/uc/ai_assistant.md | 202 ----------------------------------------- 1 file changed, 202 deletions(-) delete mode 100644 src/uc/ai_assistant.md diff --git a/src/uc/ai_assistant.md b/src/uc/ai_assistant.md deleted file mode 100644 index 3ba2a0a..0000000 --- a/src/uc/ai_assistant.md +++ /dev/null @@ -1,202 +0,0 @@ -This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. - -**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTLs for hashes - ideal for rate limiting to ensure fair resource usage. - -### Architecture overview -| Layer | Description | Data type | -| ---------- | ---------- | ---------- | -| `Session History`| `Recent conversation context` | List | -| `Rate Limiting` | `Per-user request throttling` | Hash | -| `User Memory` | `Long-term facts and preferences` | Hash | - -### Session history -AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. - -```redis:[run_confirmation=true] Store conversation history -// Add user message to session -LPUSH user:alice:history:session_001 '{"type": "human", "content": "What is the weather like?", "timestamp": 1717935001}' - -// Add AI response -LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It is sunny with 75°F temperature.", "timestamp": 1717935002}' - -// Add another user message -LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bring an umbrella?", "timestamp": 1717935003}' - -// Add AI response -LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' -``` -### Reading conversation history -You can retrieve recent messages to provide context to the AI. - -```redis:[run_confirmation=no] Read conversation history -// Get the 5 most recent messages -LRANGE user:alice:history:session_001 0 4 - -// Get the full session -LRANGE user:alice:history:session_001 0 -1 -``` -You may want to limit the size of history to retain only the N most recent items. Use LTRIM: -```redis:[run_confirmation=yes] Read conversation history -LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items -``` - -### Session expiration -Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. - -```redis:[run_confirmation=true] Session expiration -// Set session to expire in 24 hours -EXPIRE user:alice:history:session_001 86400 - -// Set session to expire in 1 hour -EXPIRE user:alice:history:session_001 3600 - -// Check remaining TTL -TTL user:alice:history:session_001 - -// Remove expiration (make persistent) -PERSIST user:alice:history:session_001 -``` - -### Rate limiting -Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTLs applied using `HSETEX` are ideal for this. - -```redis:[run_confirmation=true] Initialize Rate Limiting -// On first request - set counter with 1-minute TTL -HSETEX user:alice:rate_limit EX 60 FIELDS 1 requests_per_minute 1 -``` - -The `HINCR` command allows you to atomically increment the counter, preventing race conditions in high-concurrency scenarios. - -```redis:[run_confirmation=true] Increment Requests -// Increment request counter -HINCRBY user:alice:rate_limit requests_per_minute 1 - -// Check if field exists and get count -HEXISTS user:alice:rate_limit requests_per_minute -HGET user:alice:rate_limit requests_per_minute - -// Check TTL on the hash -TTL user:alice:rate_limit -``` -**Optionally**: if the count exceeds the allowed threshold, deny the operation. - -Different time windows serve different purposes: per-minute windows prevent burst attacks; per-hour windows prevent sustained abuse; and per-day windows enforce usage quotas. -```redis:[run_confirmation=true] Rate Limiting with Different Time Windows -// Set multiple rate limits with different TTLs -HSETEX user:alice:rate_limit EX 60 FIELDS 2 requests_per_minute 1 requests_per_hour 1 - -// Daily rate limit (24 hours) -HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 - -// Check all rate limits -HGETALL user:alice:rate_limit -``` - -### User memory (persistent preferences) -AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. - -```redis:[run_confirmation=true] Store User Preferences -// Always secure sensitive data using encryption at rest, access control (Redis ACLs), and comply with data protection laws (e.g., GDPR). -// These values are stored with embeddings to support semantic recall later using vector search. -HSET user:alice:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" - -// Storing communication preference -HSET user:alice:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - -// Store work schedule preference -HSET user:alice:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" - -// Store health information -HSET user:alice:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - -// Store pet information -HSET user:alice:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - -// Store family information -HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" -``` - -### Vector search: semantic memory recall -Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." - -Indexing persistent memory (User Memory) for semantically meaningful search. - -```redis:[run_confirmation=true] Create a Vector Index -// Create index for semantic search -FT.CREATE idx:preferences - ON HASH - PREFIX 1 user: - SCHEMA - user_id TAG - content TEXT - importance NUMERIC - timestamp NUMERIC - embedding VECTOR HNSW 6 - TYPE FLOAT32 - DIM 8 // DIM 8 is only for demonstration purposes. Real embeddings are typically 128–1536 dimensions depending on the model (e.g., sentence-transformers). - DISTANCE_METRIC COSINE -``` - -### Search for most relevant memory entries - -```redis:[run_confirmation=false] Find Top 5 Most Relevant User Memory Items -FT.SEARCH idx:preferences - "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC - DIALECT 2 -``` - -```redis:[run_confirmation=false] Search High-Importance Items Only -FT.SEARCH idx:preferences - "(@user_id:{alice} @importance:[8 +inf]) => [KNN 3 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC - DIALECT 2 -``` -```redis:[run_confirmation=false] Search Recent User Memories -FT.SEARCH idx:preferences - "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - RETURN 3 content timestamp score - SORTBY score ASC - DIALECT 2 -``` - -### Memory state monitoring -Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. -```redis:[run_confirmation=false] Monitor user sessions -// Get approximate memory usage of session -MEMORY USAGE user:alice:history:session_001 - -// Get session statistics -LLEN user:alice:history:session_001 -TTL user:alice:history:session_001 -``` -### Data cleanup -Remove all data related to a user (e.g., for GDPR compliance). - -```redis:[run_confirmation=true] Delete user data -// Remove all user data (GDPR compliance) -DEL user:alice:history:session_001 -DEL user:alice:history:session_002 -DEL user:alice:rate_limit -DEL user:alice:pref:001 -DEL user:alice:pref:002 -DEL user:alice:pref:003 -DEL user:alice:personal:001 -DEL user:alice:personal:002 -DEL user:alice:personal:003 -DEL user:alice:work:001 -DEL user:alice:work:002 -DEL user:alice:work:003 -``` - -### Next steps -Now that your assistant has memory and meaning, you can: - - Combine with RAG Pipelines - - Use sentence-transformers to generate high-dimensional vectors - - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence - - Use Redis ACLs to isolate users, enforce quotas, and monitor usage From 5c4f1afcffc17295702cfeefa85f63a1b0e19efb Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:24:27 +0200 Subject: [PATCH 40/45] Delete src/vss/ai_assistant.md --- src/vss/ai_assistant.md | 202 ---------------------------------------- 1 file changed, 202 deletions(-) delete mode 100644 src/vss/ai_assistant.md diff --git a/src/vss/ai_assistant.md b/src/vss/ai_assistant.md deleted file mode 100644 index c3e2af2..0000000 --- a/src/vss/ai_assistant.md +++ /dev/null @@ -1,202 +0,0 @@ -This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. - -**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. - -### Architecture overview -| Layer | Description | Data type | -| ---------- | ---------- | ---------- | -| `Session History`| `Recent conversation context` | List | -| `Rate Limiting` | `Per-user request throttling` | Hash | -| `User Memory` | `Long-term facts and preferences` | Hash | - -### Session history -AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. - -```redis:[run_confirmation=true] Store conversation history -// Add user message to session -LPUSH user:alice:history:session_001 '{"type": "human", "content": "What is the weather like?", "timestamp": 1717935001}' - -// Add AI response -LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It is sunny with 75°F temperature.", "timestamp": 1717935002}' - -// Add another user message -LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bring an umbrella?", "timestamp": 1717935003}' - -// Add AI response -LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' -``` -### Reading conversation history -You can retrieve recent messages to provide context to the AI. - -```redis:[run_confirmation=no] Read conversation history -// Get the 5 most recent messages -LRANGE user:alice:history:session_001 0 4 - -// Get the full session -LRANGE user:alice:history:session_001 0 -1 -``` -You may want to limit the size of history to retain only the N most recent items. Use LTRIM: -```redis:[run_confirmation=yes] Read conversation history -LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items -``` - -### Session expiration -Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. - -```redis:[run_confirmation=true] Session expiration -// Set session to expire in 24 hours -EXPIRE user:alice:history:session_001 86400 - -// Set session to expire in 1 hour -EXPIRE user:alice:history:session_001 3600 - -// Check remaining TTL -TTL user:alice:history:session_001 - -// Remove expiration (make persistent) -PERSIST user:alice:history:session_001 -``` - -### Rate limiting -Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. - -```redis:[run_confirmation=true] Initialize Rate Limiting -// On first request - set counter with 1-minute TTL -HSETEX user:alice:rate_limit EX 60 FIELDS 1 requests_per_minute 1 -``` - -The `HINCR` command allows you to atomically increment the counter, preventing race conditions in high-concurrency scenarios. - -```redis:[run_confirmation=true] Increment Requests -// Increment request counter -HINCRBY user:alice:rate_limit requests_per_minute 1 - -// Check if field exists and get count -HEXISTS user:alice:rate_limit requests_per_minute -HGET user:alice:rate_limit requests_per_minute - -// Check TTL on the hash -TTL user:alice:rate_limit -``` -**Optionally**: if the count exceeds the allowed threshold, deny the operation. - -Different time windows serve different purposes - per-minute prevents burst attacks, per-hour prevents sustained abuse, per-day enforces usage quotas. -```redis:[run_confirmation=true] Rate Limiting with Different Time Windows -// Set multiple rate limits with different TTLs -HSETEX user:alice:rate_limit EX 60 FIELDS 2 requests_per_minute 1 requests_per_hour 1 - -// Daily rate limit (24 hours) -HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 - -// Check all rate limits -HGETALL user:alice:rate_limit -``` - -### User memory (persistent preferences) -AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. - -```redis:[run_confirmation=true] Store User Preferences -// Always secure sensitive data using encryption at rest, access control (Redis ACLs), and comply with data protection laws (e.g., GDPR). -// These values are stored with embeddings to support semantic recall later using vector search. -HSET user:alice:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" - -// Storing communication preference -HSET user:alice:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - -// Store work schedule preference -HSET user:alice:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" - -// Store health information -HSET user:alice:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - -// Store pet information -HSET user:alice:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - -// Store family information -HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" -``` - -### Vector search: semantic memory recall -Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." - -Indexing persistent memory (User Memory) for semantically meaningful search. - -```redis:[run_confirmation=true] Create a Vector Index -// Create index for semantic search -FT.CREATE idx:preferences - ON HASH - PREFIX 1 user: - SCHEMA - user_id TAG - content TEXT - importance NUMERIC - timestamp NUMERIC - embedding VECTOR HNSW 6 - TYPE FLOAT32 - DIM 8 // DIM 8 is only for demonstration purposes. Real embeddings are typically 128–1536 dimensions depending on the model (e.g., sentence-transformers). - DISTANCE_METRIC COSINE -``` - -### Search for most relevant memory entries - -```redis:[run_confirmation=false] Find Top 5 Most Relevant User Memory Items -FT.SEARCH idx:preferences - "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC - DIALECT 2 -``` - -```redis:[run_confirmation=false] Search High-Importance Items Only -FT.SEARCH idx:preferences - "(@user_id:{alice} @importance:[8 +inf]) => [KNN 3 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC - DIALECT 2 -``` -```redis:[run_confirmation=false] Search Recent User Memories -FT.SEARCH idx:preferences - "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - RETURN 3 content timestamp score - SORTBY score ASC - DIALECT 2 -``` - -### Memory state monitoring -Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. -```redis:[run_confirmation=false] Monitor user sessions -// Get approximate memory usage of session -MEMORY USAGE user:alice:history:session_001 - -// Get session statistics -LLEN user:alice:history:session_001 -TTL user:alice:history:session_001 -``` -### Data cleanup -Remove all data related to a user (e.g., for GDPR compliance). - -```redis:[run_confirmation=true] Delete user data -// Remove all user data (GDPR compliance) -DEL user:alice:history:session_001 -DEL user:alice:history:session_002 -DEL user:alice:rate_limit -DEL user:alice:pref:001 -DEL user:alice:pref:002 -DEL user:alice:pref:003 -DEL user:alice:personal:001 -DEL user:alice:personal:002 -DEL user:alice:personal:003 -DEL user:alice:work:001 -DEL user:alice:work:002 -DEL user:alice:work:003 -``` - -### Next steps -Now that your assistant has memory and meaning, you can: - - Combine with RAG Pipelines - - Use sentence-transformers to generate high-dimensional vectors - - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence - - Use Redis ACLs to isolate users, enforce quotas, and monitor usage From fbf3e898aa5d8ac182a3c1f3eb7bb43701726193 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:25:18 +0200 Subject: [PATCH 41/45] Delete src/uc/personalized_recommendations.md --- src/uc/personalized_recommendations.md | 164 ------------------------- 1 file changed, 164 deletions(-) delete mode 100644 src/uc/personalized_recommendations.md diff --git a/src/uc/personalized_recommendations.md b/src/uc/personalized_recommendations.md deleted file mode 100644 index fccbae4..0000000 --- a/src/uc/personalized_recommendations.md +++ /dev/null @@ -1,164 +0,0 @@ -Imagine building a movie recommendation app that goes beyond keyword matching. Your users get intuitive, meaningful suggestions based on the true meaning of movie plots — powered by semantic understanding. - -### Store Movie Documents with Vector Embeddings -Semantic search uses vector embeddings — numeric representations of text that capture meaning, enabling search by intent rather than keywords. - -We'll import a dataset of plot summaries, each paired with an embedding vector. - -Each movie is stored as a JSON document with: - - `title`, `genres`, `year`, `plot` - - `embedding`: a binary-encoded `FLOAT32[]` for vector similarity, generated via sentence-transformer models or similar. - -```redis:[run_confirmation=true] Upload Movies -// Demo uses 8 DIM embeddings; production typically uses 128–1536D. -JSON.SET movie:001 $ '{"title":"Toy Story","genres":["Animation","Comedy","Family"],"plot":"Toys come to life when humans arent around.","year":1995,"embedding":[0.22,0.04,0.33,0.12,-0.02,0.17,0.09,0.01]}' -JSON.SET movie:002 $ '{"title":"Inside Out","genres":["Animation","Comedy","Drama"],"plot":"Emotions guide a young girl through change.","year":2015,"embedding":[0.20,0.03,0.31,0.11,-0.03,0.16,0.08,0.02]}' -JSON.SET movie:003 $ '{"title":"Whiplash","genres":["Drama","Music"],"plot":"A young drummer is pushed to greatness.","year":2014,"embedding":[0.14,0.01,0.22,0.08,-0.07,0.10,0.04,0.00]}' -JSON.SET movie:004 $ '{"title":"La La Land","genres":["Drama","Music","Romance"],"plot":"A jazz musician falls in love in LA.","year":2016,"embedding":[0.15,0.03,0.23,0.09,-0.08,0.14,0.06,0.01]}' -JSON.SET movie:005 $ '{"title":"The Matrix","genres":["Action","Sci-Fi"],"plot":"A hacker discovers reality is a simulation.","year":1999,"embedding":[0.12,-0.03,0.25,0.04,-0.10,0.09,0.05,-0.02]}' -JSON.SET movie:006 $ '{"title":"Inception","genres":["Action","Adventure","Sci-Fi"],"plot":"A thief steals information through dreams.","year":2010,"embedding":[0.14,-0.01,0.27,0.06,-0.09,0.10,0.04,-0.03]}' -JSON.SET movie:007 $ '{"title":"Tenet","genres":["Action","Sci-Fi","Thriller"],"plot":"Time-inversion to prevent World War III.","year":2020,"embedding":[0.13,-0.06,0.29,0.05,-0.11,0.12,0.06,-0.01]}' -JSON.SET movie:008 $ '{"title":"Finding Nemo","genres":["Animation","Adventure","Family"],"plot":"A clownfish searches for his son.","year":2003,"embedding":[0.18,0.02,0.30,0.10,-0.05,0.15,0.07,0.01]}' -JSON.SET movie:009 $ '{"title":"Coco","genres":["Animation","Family","Music"],"plot":"A boy enters the Land of the Dead.","year":2017,"embedding":[0.21,0.04,0.34,0.13,-0.02,0.19,0.10,0.02]}' -JSON.SET movie:010 $ '{"title":"Soul","genres":["Animation","Adventure","Comedy"],"plot":"A jazz musician explores the afterlife.","year":2020,"embedding":[0.16,0.02,0.28,0.10,-0.06,0.13,0.07,0.00]}' -JSON.SET movie:011 $ '{"title":"The Dark Knight","genres":["Action","Crime","Drama"],"plot":"Batman fights the Joker.","year":2008,"embedding":[0.12,-0.03,0.25,0.04,-0.09,0.10,0.05,-0.02]}' -JSON.SET movie:012 $ '{"title":"Frozen","genres":["Animation","Adventure","Comedy"],"plot":"A princess sets off to find her sister.","year":2013,"embedding":[0.22,0.04,0.33,0.12,-0.03,0.18,0.10,0.02]}' -JSON.SET movie:013 $ '{"title":"The Lion King","genres":["Animation","Adventure","Drama"],"plot":"A lion prince flees and returns.","year":1994,"embedding":[0.19,0.02,0.32,0.10,-0.04,0.18,0.09,0.03]}' -JSON.SET movie:014 $ '{"title":"Shrek","genres":["Animation","Adventure","Comedy"],"plot":"An ogre rescues a princess.","year":2001,"embedding":[0.21,0.03,0.32,0.11,-0.04,0.17,0.09,0.01]}' -JSON.SET movie:015 $ '{"title":"The Social Network","genres":["Biography","Drama"],"plot":"The rise of Facebook and its creator.","year":2010,"embedding":[0.10,-0.01,0.21,0.05,-0.07,0.06,0.03,-0.02]}' -JSON.SET movie:016 $ '{"title":"Guardians of the Galaxy","genres":["Action","Adventure","Sci-Fi"],"plot":"A group of intergalactic criminals must save the universe.","year":2014,"embedding":[0.13,0.00,0.28,0.07,-0.08,0.11,0.05,-0.01]}' -JSON.SET movie:017 $ '{"title":"Moana","genres":["Animation","Adventure","Family"],"plot":"A young girl sets sail to save her island.","year":2016,"embedding":[0.20,0.03,0.33,0.12,-0.03,0.17,0.09,0.02]}' -JSON.SET movie:018 $ '{"title":"Whale Rider","genres":["Drama","Family"],"plot":"A girl fights tradition to become chief.","year":2002,"embedding":[0.15,0.01,0.25,0.09,-0.05,0.12,0.06,0.00]}' -JSON.SET movie:019 $ '{"title":"Rocketman","genres":["Biography","Drama","Music"],"plot":"The story of Elton Johns breakthrough years.","year":2019,"embedding":[0.14,0.01,0.22,0.07,-0.06,0.11,0.05,0.01]}' -JSON.SET movie:020 $ '{"title":"Amadeus","genres":["Biography","Drama","Music"],"plot":"The rivalry between Mozart and Salieri.","year":1984,"embedding":[0.13,0.00,0.20,0.06,-0.07,0.10,0.04,0.00]}' -JSON.SET movie:021 $ '{"title":"The Sound of Music","genres":["Biography","Drama","Music"],"plot":"A governess brings music to a family.","year":1965,"embedding":[0.14,0.02,0.21,0.07,-0.06,0.11,0.05,0.01]}' -JSON.SET movie:022 $ '{"title":"Les Miserables","genres":["Drama","Music","Romance"],"plot":"The struggles of ex-convict Jean Valjean.","year":2012,"embedding":[0.13,0.01,0.23,0.08,-0.07,0.12,0.05,0.01]}' -JSON.SET movie:023 $ '{"title":"The Greatest Showman","genres":["Biography","Drama","Music"],"plot":"The story of P.T. Barnum and his circus.","year":2017,"embedding":[0.15,0.03,0.25,0.09,-0.05,0.13,0.06,0.02]}' -JSON.SET movie:024 $ '{"title":"A Star Is Born","genres":["Drama","Music","Romance"],"plot":"A musician helps a young singer find fame.","year":2018,"embedding":[0.14,0.02,0.24,0.08,-0.06,0.12,0.05,0.01]}' -JSON.SET movie:025 $ '{"title":"Mad Max: Fury Road","genres":["Action","Adventure","Sci-Fi"],"plot":"In a post-apocalyptic wasteland, Max helps rebels escape.","year":2015,"embedding":[0.11,-0.02,0.26,0.05,-0.10,0.08,0.05,-0.02]}' -JSON.SET movie:026 $ '{"title":"Blade Runner 2049","genres":["Sci-Fi","Thriller"],"plot":"A new blade runner uncovers secrets.","year":2017,"embedding":[0.12,-0.03,0.27,0.06,-0.09,0.09,0.06,-0.01]}' -JSON.SET movie:027 $ '{"title":"Arrival","genres":["Drama","Sci-Fi","Thriller"],"plot":"A linguist communicates with aliens.","year":2016,"embedding":[0.13,-0.01,0.28,0.07,-0.08,0.11,0.05,-0.01]}' -JSON.SET movie:028 $ '{"title":"Interstellar","genres":["Adventure","Drama","Sci-Fi"],"plot":"Explorers travel through a wormhole in space.","year":2014,"embedding":[0.14,-0.02,0.29,0.08,-0.09,0.12,0.06,-0.02]}' -JSON.SET movie:029 $ '{"title":"E.T. the Extra-Terrestrial","genres":["Family","Sci-Fi"],"plot":"A boy befriends an alien.","year":1982,"embedding":[0.17,0.01,0.31,0.10,-0.06,0.15,0.07,0.01]}' -JSON.SET movie:030 $ '{"title":"The Avengers","genres":["Action","Adventure","Sci-Fi"],"plot":"Superheroes team up to save the world.","year":2012,"embedding":[0.13,0.00,0.27,0.07,-0.08,0.11,0.06,-0.01]}' -JSON.SET movie:031 $ '{"title":"Guardians of the Galaxy Vol. 2","genres":["Action","Adventure","Comedy"],"plot":"The Guardians fight to protect the galaxy.","year":2017,"embedding":[0.15,0.01,0.28,0.09,-0.07,0.13,0.07,0.01]}' -JSON.SET movie:032 $ '{"title":"Up","genres":["Animation","Adventure","Comedy"],"plot":"An old man goes on an adventure in his flying house.","year":2009,"embedding":[0.21,0.04,0.32,0.11,-0.04,0.16,0.09,0.02]}' -JSON.SET movie:033 $ '{"title":"Zootopia","genres":["Animation","Adventure","Comedy"],"plot":"A bunny cop solves a mystery in a city of animals.","year":2016,"embedding":[0.20,0.03,0.31,0.10,-0.05,0.15,0.08,0.01]}' -JSON.SET movie:034 $ '{"title":"Big Hero 6","genres":["Animation","Action","Comedy"],"plot":"A robotics prodigy teams with friends to fight crime.","year":2014,"embedding":[0.19,0.02,0.30,0.09,-0.05,0.14,0.08,0.01]}' -JSON.SET movie:035 $ '{"title":"The Prestige","genres":["Drama","Mystery","Sci-Fi"],"plot":"Two magicians engage in a deadly rivalry.","year":2006,"embedding":[0.12,-0.02,0.24,0.06,-0.08,0.10,0.05,-0.01]}' -JSON.SET movie:036 $ '{"title":"Dunkirk","genres":["Action","Drama","History"],"plot":"Allied soldiers are evacuated during WWII.","year":2017,"embedding":[0.10,-0.03,0.22,0.05,-0.09,0.07,0.04,-0.02]}' -JSON.SET movie:037 $ '{"title":"Jumanji: Welcome to the Jungle","genres":["Action","Adventure","Comedy"],"plot":"Teens trapped in a video game jungle.","year":2017,"embedding":[0.16,0.01,0.27,0.08,-0.06,0.12,0.06,0.01]}' -JSON.SET movie:038 $ '{"title":"Cinderella","genres":["Animation","Family","Fantasy"],"plot":"A young girl overcomes her cruel stepmother.","year":1950,"embedding":[0.19,0.03,0.31,0.11,-0.04,0.16,0.08,0.02]}' -JSON.SET movie:039 $ '{"title":"Mulan","genres":["Animation","Adventure","Drama"],"plot":"A young woman disguises as a soldier.","year":1998,"embedding":[0.20,0.03,0.32,0.11,-0.04,0.17,0.09,0.02]}' -JSON.SET movie:040 $ '{"title":"Beauty and the Beast","genres":["Animation","Family","Fantasy"],"plot":"A young woman falls in love with a beast.","year":1991,"embedding":[0.18,0.02,0.30,0.10,-0.05,0.15,0.08,0.01]}' -JSON.SET movie:041 $ '{"title":"The Godfather","genres":["Crime","Drama"],"plot":"The aging patriarch of an organized crime dynasty transfers control to his son.","year":1972,"embedding":[0.11,-0.04,0.24,0.06,-0.10,0.07,0.05,-0.03]}' -JSON.SET movie:042 $ '{"title":"Pulp Fiction","genres":["Crime","Drama"],"plot":"The lives of two mob hitmen, a boxer, and others intertwine.","year":1994,"embedding":[0.12,-0.03,0.23,0.07,-0.09,0.09,0.04,-0.01]}' -JSON.SET movie:043 $ '{"title":"Forrest Gump","genres":["Drama","Romance"],"plot":"The presidencies of Kennedy and Johnson through the eyes of Forrest.","year":1994,"embedding":[0.14,0.01,0.26,0.08,-0.07,0.11,0.06,0.01]}' -JSON.SET movie:044 $ '{"title":"Gladiator","genres":["Action","Drama"],"plot":"A former Roman General seeks revenge.","year":2000,"embedding":[0.13,0.00,0.25,0.07,-0.08,0.10,0.05,0.00]}' -JSON.SET movie:045 $ '{"title":"Titanic","genres":["Drama","Romance"],"plot":"A seventeen-year-old aristocrat falls in love with a kind but poor artist.","year":1997,"embedding":[0.15,0.02,0.28,0.09,-0.06,0.13,0.06,0.01]}' -JSON.SET movie:046 $ '{"title":"Jurassic Park","genres":["Adventure","Sci-Fi"],"plot":"Scientists clone dinosaurs for a theme park.","year":1993,"embedding":[0.14,-0.01,0.26,0.08,-0.07,0.11,0.06,0.00]}' -JSON.SET movie:047 $ '{"title":"The Shawshank Redemption","genres":["Drama"],"plot":"Two imprisoned men bond over a number of years.","year":1994,"embedding":[0.15,0.00,0.27,0.09,-0.06,0.12,0.07,0.01]}' -JSON.SET movie:048 $ '{"title":"Fight Club","genres":["Drama"],"plot":"An insomniac and a soap maker form an underground fight club.","year":1999,"embedding":[0.13,-0.02,0.24,0.07,-0.08,0.10,0.05,-0.01]}' -JSON.SET movie:049 $ '{"title":"The Silence of the Lambs","genres":["Crime","Drama","Thriller"],"plot":"A young FBI cadet seeks help from an imprisoned cannibal.","year":1991,"embedding":[0.11,-0.03,0.22,0.06,-0.09,0.08,0.04,-0.02]}' -JSON.SET movie:050 $ '{"title":"The Departed","genres":["Crime","Drama","Thriller"],"plot":"An undercover cop and a mole in the police attempt to identify each other.","year":2006,"embedding":[0.12,-0.02,0.23,0.07,-0.08,0.09,0.05,-0.01]}' -JSON.SET movie:51 $ '{"title":"Saturday Night Fever","year":1977,"genres":["Drama","Music"],"plot":"A young man finds escape from his mundane life through disco dancing.","embedding":[0.154,-0.050,0.300,0.150,-0.150,0.154,0.034,-0.118]}' -JSON.SET movie:52 $ '{"title":"The Rose","year":1979,"genres":["Music","Drama"],"plot":"A rock star struggles with fame, addiction, and love.","embedding":[0.144,-0.055,0.295,0.145,-0.160,0.150,0.030,-0.120]}' -JSON.SET movie:53 $ '{"title":"Cabaret","year":1972,"genres":["Drama","Music"],"plot":"A performer and a writer navigate love and politics in pre-WWII Berlin.","embedding":[0.151,-0.052,0.297,0.148,-0.152,0.150,0.031,-0.121]}' -JSON.SET movie:54 $ '{"title":"Tommy","year":1975,"genres":["Drama","Music","Fantasy"],"plot":"A deaf and blind boy becomes a pinball champion and religious figure.","embedding":[0.149,-0.051,0.301,0.149,-0.151,0.153,0.033,-0.119]}' -JSON.SET movie:55 $ '{"title":"All That Jazz","year":1979,"genres":["Drama","Music"],"plot":"A choreographer reflects on his life and art while facing death.","embedding":[0.153,-0.049,0.299,0.151,-0.149,0.152,0.032,-0.117]}' -``` - -### Create a Vector-Enabled Search Index -Redis stores movie data as JSON documents with text fields (title, genres, plot) and vector embeddings. Indexing enables fast filtering and approximate nearest neighbor (ANN) searches on embeddings. - -```redis:[run_confirmation=true] Create a Vector Index -FT.CREATE idx:movies ON JSON PREFIX 1 "movie:" SCHEMA - $.title AS title TEXT - $.genres[*] AS genres TAG - $.plot AS plot TEXT - $.year AS year NUMERIC - $.embedding AS embedding VECTOR FLAT - 6 - TYPE FLOAT32 - DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. - DISTANCE_METRIC COSINE // COSINE = measures semantic closeness - ``` - -This sets the stage for combined textual and semantic search. - -### Semantic Search by Query Embedding -When users search “I want a fun animated movie about toys and friendship”, sentence-transformers models can convert the text into a vector. You can store and query this vector in Redis for semantic search. - -```redis:[run_confirmation=false] Search Per Plot -FT.SEARCH idx:movies "*=>[KNN 3 @embedding $vec AS score]" - PARAMS 2 vec "\x9a\x99\x19\x3f\xcd\xcc\xcc\x3d\x9a\x99\x4c\x3f\x9a\x99\x33\x3e\x9a\x99\x33\x3f\xcd\xcc\x66\x3e\xcd\xcc\xcc\x3d\xcd\xcc\x4c\x3e" - SORTBY score - RETURN 3 title plot score - DIALECT 2 -``` - -Redis returns top movies with embeddings close to the query vector - Toy Story ranks first, even if keywords don’t exactly match. - -### Adding Filters for Hybrid Search - -Now, let’s find music movies: -“A feel-good film about music and students.” - -You can combine a genre filter with vector similarity: - -```redis:[run_confirmation=false] Search Per Genre -FT.SEARCH idx:movies "@genres:{Music} =>[KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" - SORTBY score - RETURN 3 title genres score - DIALECT 2 -``` - -This hybrid query uses Redis’s tagging system plus vector search, improving relevance. - -### Using Embeddings of Existing Movies for Recommendations - -Let’s say you love Inception and want similar movies. Let’s retrieve the Inception plot embedding and use it as the query vector: -```redis:[run_confirmation=false] Get the Embedding From the Movie Document -JSON.GET movie:006 $.embedding -``` -Now let’s run vector similarity search using that embedding as a binary blob (`FLOAT32`-packed): - -```redis:[run_confirmation=false] Search for Similar Movies -FT.SEARCH idx:movies "*=>[KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\xCD\xCC\x56\x3E\x9A\x99\xF3\xBC\xCD\xCC\x00\x3F\x66\x66\x34\x3E\xC6\xF5\x1B\xBE\x9A\x99\x4D\x3E\x9A\x99\x99\x3D\x9A\x99\xB5\xBD" - SORTBY score - RETURN 2 title score - DIALECT 2 -``` - -Redis finds movies like Arrival and The Departed, showcasing content-based recommendation with semantic similarity. - -### Combining Metadata Filters with Vector Search -Let’s find classic musical rebellion films from the 90s: -```redis:[run_confirmation=false] Search for Classical Musical Rebellion Films -FT.SEARCH idx:movies "(@genres:{Music} @year:[1970 1979]) =>[KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" - SORTBY score - RETURN 4 title year genres score - DIALECT 2 -``` -This shows how Redis vector search works seamlessly with numeric and tag filters. - -### Personalizing Recommendations -Say we like Animated and Sci-Fi movies. You can personalize results by filtering the vector search: -```redis:[run_confirmation=false] Search Per Genres -FT.SEARCH idx:movies '@genres:{"Animated"|"Sci-Fi"} =>[KNN 5 @embedding $vec AS score]' - PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" - SORTBY score - RETURN 3 title genres score - DIALECT 2 -``` - -This makes Redis recommendations responsive to evolving user preferences without retraining embeddings. - -### Next Steps - - Learn more about building personalized recommendations with this [workshop](https://github.com/redis-developer/redis-movies-searcher/tree/springio-2025-workshop). - - Build a UI for natural language queries that delivers instant semantic recommendations. - - Add personalization by merging user preferences with semantic search. - - Explore advanced vector search methods like HNSW indexing for large datasets. \ No newline at end of file From 22934e366f41ea3444f9c7b7a18f0f7053488e99 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:27:01 +0200 Subject: [PATCH 42/45] Update personalized_recommendations.md --- src/vss/personalized_recommendations.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/src/vss/personalized_recommendations.md b/src/vss/personalized_recommendations.md index c0f8442..c3e2af2 100644 --- a/src/vss/personalized_recommendations.md +++ b/src/vss/personalized_recommendations.md @@ -2,14 +2,14 @@ This tutorial demonstrates how to build an AI assistant's memory system with Red **Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. -### Architecture Overview +### Architecture overview | Layer | Description | Data type | | ---------- | ---------- | ---------- | | `Session History`| `Recent conversation context` | List | | `Rate Limiting` | `Per-user request throttling` | Hash | | `User Memory` | `Long-term facts and preferences` | Hash | -### Session History +### Session history AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. ```redis:[run_confirmation=true] Store conversation history @@ -25,7 +25,7 @@ LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bri // Add AI response LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' ``` -### Reading Conversation History +### Reading conversation history You can retrieve recent messages to provide context to the AI. ```redis:[run_confirmation=no] Read conversation history @@ -40,7 +40,7 @@ You may want to limit the size of history to retain only the N most recent items LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items ``` -### Session Expiration +### Session expiration Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. ```redis:[run_confirmation=true] Session expiration @@ -57,7 +57,7 @@ TTL user:alice:history:session_001 PERSIST user:alice:history:session_001 ``` -### Rate Limiting +### Rate limiting Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. ```redis:[run_confirmation=true] Initialize Rate Limiting @@ -92,7 +92,7 @@ HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 HGETALL user:alice:rate_limit ``` -### User Memory (Persistent Preferences) +### User memory (persistent preferences) AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. ```redis:[run_confirmation=true] Store User Preferences @@ -116,7 +116,7 @@ HSET user:alice:personal:002 user_id "alice" content "has a golden retriever nam HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" ``` -### Vector Search: Semantic Memory Recall +### Vector search: semantic memory recall Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." Indexing persistent memory (User Memory) for semantically meaningful search. @@ -165,7 +165,7 @@ FT.SEARCH idx:preferences DIALECT 2 ``` -### Memory State Monitoring +### Memory state monitoring Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. ```redis:[run_confirmation=false] Monitor user sessions // Get approximate memory usage of session @@ -175,7 +175,7 @@ MEMORY USAGE user:alice:history:session_001 LLEN user:alice:history:session_001 TTL user:alice:history:session_001 ``` -### Data Cleanup +### Data cleanup Remove all data related to a user (e.g., for GDPR compliance). ```redis:[run_confirmation=true] Delete user data @@ -194,7 +194,7 @@ DEL user:alice:work:002 DEL user:alice:work:003 ``` -### Next Steps +### Next steps Now that your assistant has memory and meaning, you can: - Combine with RAG Pipelines - Use sentence-transformers to generate high-dimensional vectors From 8d4108dfff4555323c2549bee684573f1fa805b6 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:31:56 +0200 Subject: [PATCH 43/45] Create ai_assistant.md --- src/vss/ai_assistant.md | 150 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 src/vss/ai_assistant.md diff --git a/src/vss/ai_assistant.md b/src/vss/ai_assistant.md new file mode 100644 index 0000000..e6dc9bd --- /dev/null +++ b/src/vss/ai_assistant.md @@ -0,0 +1,150 @@ +This intelligent AI assistant is designed to support real-world, multi-session use with Redis as its memory core. + +What you get: + - **Smart Memory**: Ephemeral context that expires automatically, long-term facts retained forever + - **Semantic Search**: Recall relevant info by meaning using vector search + - **Zero Maintenance**: Auto-expiring short-term memory without a need to track timestamps manually + - **Multi-User**: Isolated memory per user + - **Learning**: Assistant can "understand" each user better the more it's used + +**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes, which is ideal for managing short-term memory with precision. + +### Architecture overview +| Layer | Description | +| ---------- | ---------- | +| `Working Memory`| `Short-term chat context (ephemeral)` | +| `Knowledge Base` | `Persistent facts, user preferences` | +| `Vector Search` | `Unified semantic recall across both layers` | + +### Working memory (ephemeral) +Stores recent user messages with TTL based on importance. Automatically expires to prevent bloat. +This uses `HSETEX`, a Redis 8 command that adds field-level expiration to hashes. It allows storing all temporary messages in a single key while managing TTLs per message, simplifying short-term memory management without needing multiple keys. + +```redis:[run_confirmation=true] Recent Conversations with TTL Based on Importance. +// Quick exchanges (5 min) +HSETEX user:alice:session:001 EX 300 FIELDS 1 msg:001 "What's the weather?" +// Session context (30 min) +HSETEX user:alice:session:001 EX 1800 FIELDS 1 msg:002 "I need a dentist appointment" +// Important decisions (2 hours) +HSETEX user:alice:session:001 EX 7200 FIELDS 1 msg:003 "Book it for Tuesday 2 PM" +``` + +### Knowledge base (persistent) +Long-term memory: stores important facts, user preferences, and context across sessions. These never expire. +`embedding` is a binary-encoded `FLOAT32[]` used for vector similarity that can be generated using sentence-transformers or similar libraries. Demo uses 8-dim vectors; production models typically use 128–1536 dimensions. + +```redis:[run_confirmation=true] Important User Information That Never Expires. +// User preferences - need vector fields for search +HSET user:alice:knowledge:pref:001 user_id "alice" memory_type "knowledge" content "prefers mornings before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" +HSET user:alice:knowledge:pref:002 user_id "alice" memory_type "knowledge" content "likes detailed explanations" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +// Personal facts +HSET user:alice:knowledge:personal:001 user_id "alice" memory_type "knowledge" content "allergic to shellfish" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +HSET user:alice:knowledge:personal:002 user_id "alice" memory_type "knowledge" content "golden retriever named Max" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" +// Work context +HSET user:alice:knowledge:work:001 user_id "alice" memory_type "knowledge" content "Senior PM at TechCorp" importance 8 timestamp 1717935000 embedding "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" +HSET user:alice:knowledge:work:002 user_id "alice" memory_type "knowledge" content "leading Project Apollo" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x80\x00\x00\x3f\x40\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" +``` + +### Vector search: semantic memory recall +Unify working + knowledge memory into a vector index for semantically meaningful search. + +```redis:[run_confirmation=true] Create a Vector Index +FT.CREATE idx:memory + ON HASH + PREFIX 2 vmemory: user: + SCHEMA + user_id TAG + memory_type TAG + content TEXT + importance NUMERIC + timestamp NUMERIC + embedding VECTOR HNSW 6 + TYPE FLOAT32 + DIM 8 // DIM = embedding size; 8 used here for simplicity — in production, use 128 to 1536 + DISTANCE_METRIC COSINE // COSINE = measures semantic closeness +``` + +### Add indexed memory +Populate the vector index with memory items from both ephemeral and persistent layers. + +```redis:[run_confirmation=true] Add entries for the chatbot +// Working memory (expires from Redis, stays in search until rebuild) +HSET vmemory:alice:001 user_id "alice" memory_type "working" content "Book dentist for Tuesday 2 PM" importance 8 timestamp 1717935310 embedding "\x3f\x80\x00\x00\x40\x00\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x20\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00" + +// Knowledge base (persistent) +HSET vmemory:alice:kb:001 user_id "alice" memory_type "knowledge" content "allergic to shellfish" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +``` + +### Full memory search +Search across all memories to recall relevant data. + +```redis:[run_confirmation=false] Find Top 5 Related Messages By Meaning +FT.SEARCH idx:memory + "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" + RETURN 4 content memory_type importance score + SORTBY score ASC + DIALECT 2 +``` + +### Session-only search +Limit results to current conversation context (ephemeral memory). + +```redis:[run_confirmation=false] Session-Only Search +FT.SEARCH idx:memory + "(@user_id:{alice} @memory_type:{working}) => [KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" + RETURN 4 content score timestamp memory_type + SORTBY score ASC + DIALECT 2 +``` + +### Knowledge-only search +Focus only on persistent memory: facts, preferences, decisions. + +```redis:[run_confirmation=false] Knowledge-Only Search +FT.SEARCH idx:memory + "(@user_id:{alice} @memory_type:{knowledge}) => [KNN 8 @embedding $vec AS score]" + PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" + RETURN 4 content importance score timestamp + SORTBY score ASC + DIALECT 2 +``` + +### Monitoring memory state +Use these queries to inspect what’s stored in memory. + +```redis:[run_confirmation=false] Check Memory State +// Check active session memory +HGETALL user:alice:knowledge:pref:001 // Example for one preference item + +// View user knowledge +HGETALL user:alice:knowledge:pref:001 + +// Search user's memories +FT.SEARCH idx:memory + "@user_id:{alice}" + RETURN 3 content memory_type importance +``` + +### Data cleanup + +For privacy compliance, delete all user-related keys. + +```redis:[run_confirmation=true] Complete user removal +DEL user:alice:knowledge:pref:001 +DEL user:alice:knowledge:pref:002 +DEL user:alice:knowledge:personal:001 +DEL user:alice:knowledge:personal:002 +DEL user:alice:knowledge:work:001 +DEL user:alice:knowledge:work:002 +DEL vmemory:alice:001 +DEL vmemory:alice:kb:001 +``` + +### Next steps +Now that your assistant has memory and meaning, you can: + - Combine with RAG Pipelines + - Use sentence-transformers to generate high-dimensional vectors + - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence + - Use Redis ACLs to isolate users, enforce quotas, and monitor usage From f0d0b8bf4e545b750284a01347a71c6e37ee0f04 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Tue, 29 Jul 2025 22:39:37 +0200 Subject: [PATCH 44/45] Update personalized_recommendations.md --- src/vss/personalized_recommendations.md | 320 +++++++++++------------- 1 file changed, 141 insertions(+), 179 deletions(-) diff --git a/src/vss/personalized_recommendations.md b/src/vss/personalized_recommendations.md index c3e2af2..60180fa 100644 --- a/src/vss/personalized_recommendations.md +++ b/src/vss/personalized_recommendations.md @@ -1,202 +1,164 @@ -This tutorial demonstrates how to build an AI assistant's memory system with Redis as its memory core. - -**Note**: Requires [Redis 8](https://hub.docker.com/_/redis/tags) for `HSETEX`, which adds per-field TTL for hashes - ideal for rate limiting to ensure fair resource usage. - -### Architecture overview -| Layer | Description | Data type | -| ---------- | ---------- | ---------- | -| `Session History`| `Recent conversation context` | List | -| `Rate Limiting` | `Per-user request throttling` | Hash | -| `User Memory` | `Long-term facts and preferences` | Hash | - -### Session history -AI assistants need context from previous messages to provide coherent responses. Without conversation history, each interaction would be isolated. Redis lists are simple, ordered, and efficient for storing chat transcripts. - -```redis:[run_confirmation=true] Store conversation history -// Add user message to session -LPUSH user:alice:history:session_001 '{"type": "human", "content": "What is the weather like?", "timestamp": 1717935001}' - -// Add AI response -LPUSH user:alice:history:session_001 '{"type": "ai", "content": "It is sunny with 75°F temperature.", "timestamp": 1717935002}' - -// Add another user message -LPUSH user:alice:history:session_001 '{"type": "human", "content": "Should I bring an umbrella?", "timestamp": 1717935003}' - -// Add AI response -LPUSH user:alice:history:session_001 '{"type": "ai", "content": "No umbrella needed today!", "timestamp": 1717935004}' +Imagine building a movie recommendation app that goes beyond keyword matching. Your users get intuitive, meaningful suggestions based on the true meaning of movie plots — powered by semantic understanding. + +### Store movie documents with vector embeddings +Semantic search uses vector embeddings — numeric representations of text that capture meaning, enabling search by intent rather than keywords. + +We'll import a dataset of plot summaries, each paired with an embedding vector. + +Each movie is stored as a JSON document with: + - `title`, `genres`, `year`, `plot` + - `embedding`: a binary-encoded `FLOAT32[]` for vector similarity, generated via sentence-transformer models or similar. + +```redis:[run_confirmation=true] Upload Movies +// Demo uses 8 DIM embeddings; production typically uses 128–1536D. +JSON.SET movie:001 $ '{"title":"Toy Story","genres":["Animation","Comedy","Family"],"plot":"Toys come to life when humans arent around.","year":1995,"embedding":[0.22,0.04,0.33,0.12,-0.02,0.17,0.09,0.01]}' +JSON.SET movie:002 $ '{"title":"Inside Out","genres":["Animation","Comedy","Drama"],"plot":"Emotions guide a young girl through change.","year":2015,"embedding":[0.20,0.03,0.31,0.11,-0.03,0.16,0.08,0.02]}' +JSON.SET movie:003 $ '{"title":"Whiplash","genres":["Drama","Music"],"plot":"A young drummer is pushed to greatness.","year":2014,"embedding":[0.14,0.01,0.22,0.08,-0.07,0.10,0.04,0.00]}' +JSON.SET movie:004 $ '{"title":"La La Land","genres":["Drama","Music","Romance"],"plot":"A jazz musician falls in love in LA.","year":2016,"embedding":[0.15,0.03,0.23,0.09,-0.08,0.14,0.06,0.01]}' +JSON.SET movie:005 $ '{"title":"The Matrix","genres":["Action","Sci-Fi"],"plot":"A hacker discovers reality is a simulation.","year":1999,"embedding":[0.12,-0.03,0.25,0.04,-0.10,0.09,0.05,-0.02]}' +JSON.SET movie:006 $ '{"title":"Inception","genres":["Action","Adventure","Sci-Fi"],"plot":"A thief steals information through dreams.","year":2010,"embedding":[0.14,-0.01,0.27,0.06,-0.09,0.10,0.04,-0.03]}' +JSON.SET movie:007 $ '{"title":"Tenet","genres":["Action","Sci-Fi","Thriller"],"plot":"Time-inversion to prevent World War III.","year":2020,"embedding":[0.13,-0.06,0.29,0.05,-0.11,0.12,0.06,-0.01]}' +JSON.SET movie:008 $ '{"title":"Finding Nemo","genres":["Animation","Adventure","Family"],"plot":"A clownfish searches for his son.","year":2003,"embedding":[0.18,0.02,0.30,0.10,-0.05,0.15,0.07,0.01]}' +JSON.SET movie:009 $ '{"title":"Coco","genres":["Animation","Family","Music"],"plot":"A boy enters the Land of the Dead.","year":2017,"embedding":[0.21,0.04,0.34,0.13,-0.02,0.19,0.10,0.02]}' +JSON.SET movie:010 $ '{"title":"Soul","genres":["Animation","Adventure","Comedy"],"plot":"A jazz musician explores the afterlife.","year":2020,"embedding":[0.16,0.02,0.28,0.10,-0.06,0.13,0.07,0.00]}' +JSON.SET movie:011 $ '{"title":"The Dark Knight","genres":["Action","Crime","Drama"],"plot":"Batman fights the Joker.","year":2008,"embedding":[0.12,-0.03,0.25,0.04,-0.09,0.10,0.05,-0.02]}' +JSON.SET movie:012 $ '{"title":"Frozen","genres":["Animation","Adventure","Comedy"],"plot":"A princess sets off to find her sister.","year":2013,"embedding":[0.22,0.04,0.33,0.12,-0.03,0.18,0.10,0.02]}' +JSON.SET movie:013 $ '{"title":"The Lion King","genres":["Animation","Adventure","Drama"],"plot":"A lion prince flees and returns.","year":1994,"embedding":[0.19,0.02,0.32,0.10,-0.04,0.18,0.09,0.03]}' +JSON.SET movie:014 $ '{"title":"Shrek","genres":["Animation","Adventure","Comedy"],"plot":"An ogre rescues a princess.","year":2001,"embedding":[0.21,0.03,0.32,0.11,-0.04,0.17,0.09,0.01]}' +JSON.SET movie:015 $ '{"title":"The Social Network","genres":["Biography","Drama"],"plot":"The rise of Facebook and its creator.","year":2010,"embedding":[0.10,-0.01,0.21,0.05,-0.07,0.06,0.03,-0.02]}' +JSON.SET movie:016 $ '{"title":"Guardians of the Galaxy","genres":["Action","Adventure","Sci-Fi"],"plot":"A group of intergalactic criminals must save the universe.","year":2014,"embedding":[0.13,0.00,0.28,0.07,-0.08,0.11,0.05,-0.01]}' +JSON.SET movie:017 $ '{"title":"Moana","genres":["Animation","Adventure","Family"],"plot":"A young girl sets sail to save her island.","year":2016,"embedding":[0.20,0.03,0.33,0.12,-0.03,0.17,0.09,0.02]}' +JSON.SET movie:018 $ '{"title":"Whale Rider","genres":["Drama","Family"],"plot":"A girl fights tradition to become chief.","year":2002,"embedding":[0.15,0.01,0.25,0.09,-0.05,0.12,0.06,0.00]}' +JSON.SET movie:019 $ '{"title":"Rocketman","genres":["Biography","Drama","Music"],"plot":"The story of Elton Johns breakthrough years.","year":2019,"embedding":[0.14,0.01,0.22,0.07,-0.06,0.11,0.05,0.01]}' +JSON.SET movie:020 $ '{"title":"Amadeus","genres":["Biography","Drama","Music"],"plot":"The rivalry between Mozart and Salieri.","year":1984,"embedding":[0.13,0.00,0.20,0.06,-0.07,0.10,0.04,0.00]}' +JSON.SET movie:021 $ '{"title":"The Sound of Music","genres":["Biography","Drama","Music"],"plot":"A governess brings music to a family.","year":1965,"embedding":[0.14,0.02,0.21,0.07,-0.06,0.11,0.05,0.01]}' +JSON.SET movie:022 $ '{"title":"Les Miserables","genres":["Drama","Music","Romance"],"plot":"The struggles of ex-convict Jean Valjean.","year":2012,"embedding":[0.13,0.01,0.23,0.08,-0.07,0.12,0.05,0.01]}' +JSON.SET movie:023 $ '{"title":"The Greatest Showman","genres":["Biography","Drama","Music"],"plot":"The story of P.T. Barnum and his circus.","year":2017,"embedding":[0.15,0.03,0.25,0.09,-0.05,0.13,0.06,0.02]}' +JSON.SET movie:024 $ '{"title":"A Star Is Born","genres":["Drama","Music","Romance"],"plot":"A musician helps a young singer find fame.","year":2018,"embedding":[0.14,0.02,0.24,0.08,-0.06,0.12,0.05,0.01]}' +JSON.SET movie:025 $ '{"title":"Mad Max: Fury Road","genres":["Action","Adventure","Sci-Fi"],"plot":"In a post-apocalyptic wasteland, Max helps rebels escape.","year":2015,"embedding":[0.11,-0.02,0.26,0.05,-0.10,0.08,0.05,-0.02]}' +JSON.SET movie:026 $ '{"title":"Blade Runner 2049","genres":["Sci-Fi","Thriller"],"plot":"A new blade runner uncovers secrets.","year":2017,"embedding":[0.12,-0.03,0.27,0.06,-0.09,0.09,0.06,-0.01]}' +JSON.SET movie:027 $ '{"title":"Arrival","genres":["Drama","Sci-Fi","Thriller"],"plot":"A linguist communicates with aliens.","year":2016,"embedding":[0.13,-0.01,0.28,0.07,-0.08,0.11,0.05,-0.01]}' +JSON.SET movie:028 $ '{"title":"Interstellar","genres":["Adventure","Drama","Sci-Fi"],"plot":"Explorers travel through a wormhole in space.","year":2014,"embedding":[0.14,-0.02,0.29,0.08,-0.09,0.12,0.06,-0.02]}' +JSON.SET movie:029 $ '{"title":"E.T. the Extra-Terrestrial","genres":["Family","Sci-Fi"],"plot":"A boy befriends an alien.","year":1982,"embedding":[0.17,0.01,0.31,0.10,-0.06,0.15,0.07,0.01]}' +JSON.SET movie:030 $ '{"title":"The Avengers","genres":["Action","Adventure","Sci-Fi"],"plot":"Superheroes team up to save the world.","year":2012,"embedding":[0.13,0.00,0.27,0.07,-0.08,0.11,0.06,-0.01]}' +JSON.SET movie:031 $ '{"title":"Guardians of the Galaxy Vol. 2","genres":["Action","Adventure","Comedy"],"plot":"The Guardians fight to protect the galaxy.","year":2017,"embedding":[0.15,0.01,0.28,0.09,-0.07,0.13,0.07,0.01]}' +JSON.SET movie:032 $ '{"title":"Up","genres":["Animation","Adventure","Comedy"],"plot":"An old man goes on an adventure in his flying house.","year":2009,"embedding":[0.21,0.04,0.32,0.11,-0.04,0.16,0.09,0.02]}' +JSON.SET movie:033 $ '{"title":"Zootopia","genres":["Animation","Adventure","Comedy"],"plot":"A bunny cop solves a mystery in a city of animals.","year":2016,"embedding":[0.20,0.03,0.31,0.10,-0.05,0.15,0.08,0.01]}' +JSON.SET movie:034 $ '{"title":"Big Hero 6","genres":["Animation","Action","Comedy"],"plot":"A robotics prodigy teams with friends to fight crime.","year":2014,"embedding":[0.19,0.02,0.30,0.09,-0.05,0.14,0.08,0.01]}' +JSON.SET movie:035 $ '{"title":"The Prestige","genres":["Drama","Mystery","Sci-Fi"],"plot":"Two magicians engage in a deadly rivalry.","year":2006,"embedding":[0.12,-0.02,0.24,0.06,-0.08,0.10,0.05,-0.01]}' +JSON.SET movie:036 $ '{"title":"Dunkirk","genres":["Action","Drama","History"],"plot":"Allied soldiers are evacuated during WWII.","year":2017,"embedding":[0.10,-0.03,0.22,0.05,-0.09,0.07,0.04,-0.02]}' +JSON.SET movie:037 $ '{"title":"Jumanji: Welcome to the Jungle","genres":["Action","Adventure","Comedy"],"plot":"Teens trapped in a video game jungle.","year":2017,"embedding":[0.16,0.01,0.27,0.08,-0.06,0.12,0.06,0.01]}' +JSON.SET movie:038 $ '{"title":"Cinderella","genres":["Animation","Family","Fantasy"],"plot":"A young girl overcomes her cruel stepmother.","year":1950,"embedding":[0.19,0.03,0.31,0.11,-0.04,0.16,0.08,0.02]}' +JSON.SET movie:039 $ '{"title":"Mulan","genres":["Animation","Adventure","Drama"],"plot":"A young woman disguises as a soldier.","year":1998,"embedding":[0.20,0.03,0.32,0.11,-0.04,0.17,0.09,0.02]}' +JSON.SET movie:040 $ '{"title":"Beauty and the Beast","genres":["Animation","Family","Fantasy"],"plot":"A young woman falls in love with a beast.","year":1991,"embedding":[0.18,0.02,0.30,0.10,-0.05,0.15,0.08,0.01]}' +JSON.SET movie:041 $ '{"title":"The Godfather","genres":["Crime","Drama"],"plot":"The aging patriarch of an organized crime dynasty transfers control to his son.","year":1972,"embedding":[0.11,-0.04,0.24,0.06,-0.10,0.07,0.05,-0.03]}' +JSON.SET movie:042 $ '{"title":"Pulp Fiction","genres":["Crime","Drama"],"plot":"The lives of two mob hitmen, a boxer, and others intertwine.","year":1994,"embedding":[0.12,-0.03,0.23,0.07,-0.09,0.09,0.04,-0.01]}' +JSON.SET movie:043 $ '{"title":"Forrest Gump","genres":["Drama","Romance"],"plot":"The presidencies of Kennedy and Johnson through the eyes of Forrest.","year":1994,"embedding":[0.14,0.01,0.26,0.08,-0.07,0.11,0.06,0.01]}' +JSON.SET movie:044 $ '{"title":"Gladiator","genres":["Action","Drama"],"plot":"A former Roman General seeks revenge.","year":2000,"embedding":[0.13,0.00,0.25,0.07,-0.08,0.10,0.05,0.00]}' +JSON.SET movie:045 $ '{"title":"Titanic","genres":["Drama","Romance"],"plot":"A seventeen-year-old aristocrat falls in love with a kind but poor artist.","year":1997,"embedding":[0.15,0.02,0.28,0.09,-0.06,0.13,0.06,0.01]}' +JSON.SET movie:046 $ '{"title":"Jurassic Park","genres":["Adventure","Sci-Fi"],"plot":"Scientists clone dinosaurs for a theme park.","year":1993,"embedding":[0.14,-0.01,0.26,0.08,-0.07,0.11,0.06,0.00]}' +JSON.SET movie:047 $ '{"title":"The Shawshank Redemption","genres":["Drama"],"plot":"Two imprisoned men bond over a number of years.","year":1994,"embedding":[0.15,0.00,0.27,0.09,-0.06,0.12,0.07,0.01]}' +JSON.SET movie:048 $ '{"title":"Fight Club","genres":["Drama"],"plot":"An insomniac and a soap maker form an underground fight club.","year":1999,"embedding":[0.13,-0.02,0.24,0.07,-0.08,0.10,0.05,-0.01]}' +JSON.SET movie:049 $ '{"title":"The Silence of the Lambs","genres":["Crime","Drama","Thriller"],"plot":"A young FBI cadet seeks help from an imprisoned cannibal.","year":1991,"embedding":[0.11,-0.03,0.22,0.06,-0.09,0.08,0.04,-0.02]}' +JSON.SET movie:050 $ '{"title":"The Departed","genres":["Crime","Drama","Thriller"],"plot":"An undercover cop and a mole in the police attempt to identify each other.","year":2006,"embedding":[0.12,-0.02,0.23,0.07,-0.08,0.09,0.05,-0.01]}' +JSON.SET movie:51 $ '{"title":"Saturday Night Fever","year":1977,"genres":["Drama","Music"],"plot":"A young man finds escape from his mundane life through disco dancing.","embedding":[0.154,-0.050,0.300,0.150,-0.150,0.154,0.034,-0.118]}' +JSON.SET movie:52 $ '{"title":"The Rose","year":1979,"genres":["Music","Drama"],"plot":"A rock star struggles with fame, addiction, and love.","embedding":[0.144,-0.055,0.295,0.145,-0.160,0.150,0.030,-0.120]}' +JSON.SET movie:53 $ '{"title":"Cabaret","year":1972,"genres":["Drama","Music"],"plot":"A performer and a writer navigate love and politics in pre-WWII Berlin.","embedding":[0.151,-0.052,0.297,0.148,-0.152,0.150,0.031,-0.121]}' +JSON.SET movie:54 $ '{"title":"Tommy","year":1975,"genres":["Drama","Music","Fantasy"],"plot":"A deaf and blind boy becomes a pinball champion and religious figure.","embedding":[0.149,-0.051,0.301,0.149,-0.151,0.153,0.033,-0.119]}' +JSON.SET movie:55 $ '{"title":"All That Jazz","year":1979,"genres":["Drama","Music"],"plot":"A choreographer reflects on his life and art while facing death.","embedding":[0.153,-0.049,0.299,0.151,-0.149,0.152,0.032,-0.117]}' ``` -### Reading conversation history -You can retrieve recent messages to provide context to the AI. - -```redis:[run_confirmation=no] Read conversation history -// Get the 5 most recent messages -LRANGE user:alice:history:session_001 0 4 -// Get the full session -LRANGE user:alice:history:session_001 0 -1 -``` -You may want to limit the size of history to retain only the N most recent items. Use LTRIM: -```redis:[run_confirmation=yes] Read conversation history -LTRIM user:alice:history:session_001 0 29 // keep only latest 30 items -``` - -### Session expiration -Without expiration, session history will accumulate indefinitely. Expiring keys improves memory usage and ensures privacy. - -```redis:[run_confirmation=true] Session expiration -// Set session to expire in 24 hours -EXPIRE user:alice:history:session_001 86400 - -// Set session to expire in 1 hour -EXPIRE user:alice:history:session_001 3600 - -// Check remaining TTL -TTL user:alice:history:session_001 - -// Remove expiration (make persistent) -PERSIST user:alice:history:session_001 -``` - -### Rate limiting -Rate limiting prevents abuse and ensures fair usage across users. Redis hashes with field-level TTL via `HSETEX` are ideal for this. - -```redis:[run_confirmation=true] Initialize Rate Limiting -// On first request - set counter with 1-minute TTL -HSETEX user:alice:rate_limit EX 60 FIELDS 1 requests_per_minute 1 -``` +### Create a vector-enabled search index +Redis stores movie data as JSON documents with text fields (title, genres, plot) and vector embeddings. Indexing enables fast filtering and approximate nearest neighbor (ANN) searches on embeddings. -The `HINCR` command allows you to atomically increment the counter, preventing race conditions in high-concurrency scenarios. - -```redis:[run_confirmation=true] Increment Requests -// Increment request counter -HINCRBY user:alice:rate_limit requests_per_minute 1 - -// Check if field exists and get count -HEXISTS user:alice:rate_limit requests_per_minute -HGET user:alice:rate_limit requests_per_minute - -// Check TTL on the hash -TTL user:alice:rate_limit -``` -**Optionally**: if the count exceeds the allowed threshold, deny the operation. - -Different time windows serve different purposes - per-minute prevents burst attacks, per-hour prevents sustained abuse, per-day enforces usage quotas. -```redis:[run_confirmation=true] Rate Limiting with Different Time Windows -// Set multiple rate limits with different TTLs -HSETEX user:alice:rate_limit EX 60 FIELDS 2 requests_per_minute 1 requests_per_hour 1 - -// Daily rate limit (24 hours) -HSETEX user:alice:rate_limit EX 86400 FIELDS 1 requests_per_day 1 - -// Check all rate limits -HGETALL user:alice:rate_limit +```redis:[run_confirmation=true] Create a Vector Index +FT.CREATE idx:movies ON JSON PREFIX 1 "movie:" SCHEMA + $.title AS title TEXT + $.genres[*] AS genres TAG + $.plot AS plot TEXT + $.year AS year NUMERIC + $.embedding AS embedding VECTOR FLAT + 6 + TYPE FLOAT32 + DIM 8 // DIM = embedding size, DIM 8 is just for demo purposes. In real use, embeddings are usually 128–1536 dimensions. + DISTANCE_METRIC COSINE // COSINE = measures semantic closeness + ``` + +This sets the stage for combined textual and semantic search. + +### Semantic search by query embedding +When users search “I want a fun animated movie about toys and friendship”, sentence-transformers models can convert the text into a vector. You can store and query this vector in Redis for semantic search. + +```redis:[run_confirmation=false] Search Per Plot +FT.SEARCH idx:movies "*=>[KNN 3 @embedding $vec AS score]" + PARAMS 2 vec "\x9a\x99\x19\x3f\xcd\xcc\xcc\x3d\x9a\x99\x4c\x3f\x9a\x99\x33\x3e\x9a\x99\x33\x3f\xcd\xcc\x66\x3e\xcd\xcc\xcc\x3d\xcd\xcc\x4c\x3e" + SORTBY score + RETURN 3 title plot score + DIALECT 2 ``` -### User memory (persistent preferences) -AI assistants become more helpful when they remember user preferences, schedules, or relevant facts. This persistent memory enables personalization over time. - -```redis:[run_confirmation=true] Store User Preferences -// Always secure sensitive data using encryption at rest, access control (Redis ACLs), and comply with data protection laws (e.g., GDPR). -// These values are stored with embeddings to support semantic recall later using vector search. -HSET user:alice:pref:001 user_id "alice" content "prefers morning appointments before 10 AM" importance 9 timestamp 1717935000 embedding "\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00" +Redis returns top movies with embeddings close to the query vector - Toy Story ranks first, even if keywords don’t exactly match. -// Storing communication preference -HSET user:alice:pref:002 user_id "alice" content "likes detailed explanations with examples" importance 8 timestamp 1717935000 embedding "\x3f\x40\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +### Adding filters for hybrid search -// Store work schedule preference -HSET user:alice:pref:003 user_id "alice" content "works remotely on Fridays" importance 7 timestamp 1717935000 embedding "\x40\x80\x00\x00\x3f\x00\x00\x00\x40\x40\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" +Now, find music movies: +“A feel-good film about music and students.” -// Store health information -HSET user:alice:personal:001 user_id "alice" content "allergic to shellfish and nuts" importance 10 timestamp 1717935000 embedding "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" +You can combine a genre filter with vector similarity: -// Store pet information -HSET user:alice:personal:002 user_id "alice" content "has a golden retriever named Max, 3 years old" importance 7 timestamp 1717935000 embedding "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - -// Store family information -HSET user:alice:personal:003 user_id "alice" content "married to Bob, two kids Sarah (8) and Tom (5)" importance 9 timestamp 1717935000 embedding "\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x3f\x00\x00\x00" +```redis:[run_confirmation=false] Search Per Genre +FT.SEARCH idx:movies "@genres:{Music} =>[KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" + SORTBY score + RETURN 3 title genres score + DIALECT 2 ``` -### Vector search: semantic memory recall -Semantic search allows AI to retrieve relevant memory even when exact keywords don't match. For example, a query about "meetings" might return facts about "morning appointments." +This hybrid query uses Redis’s tagging system plus vector search, improving relevance. -Indexing persistent memory (User Memory) for semantically meaningful search. +### Using embeddings of existing movies for recommendations -```redis:[run_confirmation=true] Create a Vector Index -// Create index for semantic search -FT.CREATE idx:preferences - ON HASH - PREFIX 1 user: - SCHEMA - user_id TAG - content TEXT - importance NUMERIC - timestamp NUMERIC - embedding VECTOR HNSW 6 - TYPE FLOAT32 - DIM 8 // DIM 8 is only for demonstration purposes. Real embeddings are typically 128–1536 dimensions depending on the model (e.g., sentence-transformers). - DISTANCE_METRIC COSINE +You love the movie Inception and want to search for similar movies. Retrieve the Inception plot embedding and use it as the query vector: +```redis:[run_confirmation=false] Get the Embedding From the Movie Document +JSON.GET movie:006 $.embedding ``` +Now run vector similarity search using that embedding as a binary. -### Search for most relevant memory entries - -```redis:[run_confirmation=false] Find Top 5 Most Relevant User Memory Items -FT.SEARCH idx:preferences - "(@user_id:{alice}) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x00\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x60\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x3f\x00\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC +```redis:[run_confirmation=false] Search for Similar Movies +FT.SEARCH idx:movies "*=>[KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\xCD\xCC\x56\x3E\x9A\x99\xF3\xBC\xCD\xCC\x00\x3F\x66\x66\x34\x3E\xC6\xF5\x1B\xBE\x9A\x99\x4D\x3E\x9A\x99\x99\x3D\x9A\x99\xB5\xBD" + SORTBY score + RETURN 2 title score DIALECT 2 ``` -```redis:[run_confirmation=false] Search High-Importance Items Only -FT.SEARCH idx:preferences - "(@user_id:{alice} @importance:[8 +inf]) => [KNN 3 @embedding $vec AS score]" - PARAMS 2 vec "\x40\x40\x00\x00\x3f\x00\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x80\x00\x00\x40\x60\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00" - RETURN 4 content importance score timestamp - SORTBY score ASC - DIALECT 2 -``` -```redis:[run_confirmation=false] Search Recent User Memories -FT.SEARCH idx:preferences - "(@user_id:{alice} @timestamp:[1717935000 +inf]) => [KNN 5 @embedding $vec AS score]" - PARAMS 2 vec "\x3f\x80\x00\x00\x40\x40\x00\x00\x40\x00\x00\x00\x3f\x40\x00\x00\x40\x80\x00\x00\x40\x20\x00\x00\x3f\x00\x00\x00\x40\x60\x00\x00" - RETURN 3 content timestamp score - SORTBY score ASC - DIALECT 2 -``` +Redis finds movies like Arrival and The Departed, showcasing content-based recommendation with semantic similarity. -### Memory state monitoring -Understanding what's stored in memory helps debug issues, optimize performance, and ensure data quality. It's also essential for user privacy compliance. -```redis:[run_confirmation=false] Monitor user sessions -// Get approximate memory usage of session -MEMORY USAGE user:alice:history:session_001 - -// Get session statistics -LLEN user:alice:history:session_001 -TTL user:alice:history:session_001 +### Combining metadata filters with vector search +Find classic musical rebellion films from the 90s: +```redis:[run_confirmation=false] Search for Classical Musical Rebellion Films +FT.SEARCH idx:movies "(@genres:{Music} @year:[1970 1979]) =>[KNN 5 @embedding $vec AS score]" + PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" + SORTBY score + RETURN 4 title year genres score + DIALECT 2 ``` -### Data cleanup -Remove all data related to a user (e.g., for GDPR compliance). - -```redis:[run_confirmation=true] Delete user data -// Remove all user data (GDPR compliance) -DEL user:alice:history:session_001 -DEL user:alice:history:session_002 -DEL user:alice:rate_limit -DEL user:alice:pref:001 -DEL user:alice:pref:002 -DEL user:alice:pref:003 -DEL user:alice:personal:001 -DEL user:alice:personal:002 -DEL user:alice:personal:003 -DEL user:alice:work:001 -DEL user:alice:work:002 -DEL user:alice:work:003 +This shows how Redis vector search works seamlessly with numeric and tag filters. + +### Personalizing recommendations +You like Animated and Sci-Fi movies. Personalize results by filtering the vector search: +```redis:[run_confirmation=false] Search Per Genres +FT.SEARCH idx:movies '@genres:{"Animated"|"Sci-Fi"} =>[KNN 5 @embedding $vec AS score]' + PARAMS 2 vec "\x9a\x99\x1d\x3e\xcd\xcc\x4c\xbd\x9a\x99\x99\x3e\x9a\x99\x19\x3e\x9a\x99\x19\xbe\x9a\x99\x1d\x3e\xcd\xcc\x0c\x3e\x9a\x99\xf1\xbc" + SORTBY score + RETURN 3 title genres score + DIALECT 2 ``` +This makes Redis recommendations responsive to evolving user preferences without retraining embeddings. + ### Next steps -Now that your assistant has memory and meaning, you can: - - Combine with RAG Pipelines - - Use sentence-transformers to generate high-dimensional vectors - - Add [Redis Flex](https://redis.io/solutions/flex/?utm_source=redisinsight&utm_medium=app&utm_campaign=tutorials) for fallback persistence - - Use Redis ACLs to isolate users, enforce quotas, and monitor usage + - Learn more about building personalized recommendations with this [workshop](https://github.com/redis-developer/redis-movies-searcher/tree/springio-2025-workshop). + - Build a UI for natural language queries that delivers instant semantic recommendations. + - Add personalization by merging user preferences with semantic search. + - Explore advanced vector search methods like HNSW indexing for large datasets. From e2d6860eb25bab86e51cd952b01159864d959527 Mon Sep 17 00:00:00 2001 From: ViktarStarastsenka <99594890+ViktarStarastsenka@users.noreply.github.com> Date: Thu, 31 Jul 2025 23:10:18 +0200 Subject: [PATCH 45/45] Update manifest.json --- src/manifest.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/manifest.json b/src/manifest.json index fe0b65c..805a623 100644 --- a/src/manifest.json +++ b/src/manifest.json @@ -40,7 +40,7 @@ { "type": "group", "id": "vss", - "label": "Vector search explained", + "label": "Vector search examples", "args": { "initialIsOpen": false },