Introducing Synapse

What is Synapse?

I recently started using new personal knowledgebase programs that I thought would be good for what I needed to do, but I found I wanted something more like NotebookLM, but while that's great for research, it's not really a PKB at this point.  So, I came up with an idea to see if I could create my own.  I know that tools like Obsidian have this level of web/AI integration already, but I don't like having to use things like Syncthing or some other file synching system.   Here is the basic goal:

My goal of this project is to make my own personal knowledge base that can also give me results from the web that can help augment or provide insights into my notes. Like NotebookLM but also like Obsidian notes. I've got the searching for my notes, but I want to expand that into web information.

This was an ambitious project to say the least, and on top of that I wanted to have it somewhat functional within a month or so.  Here are the main features it needed to have to start.

• First and foremost, I needed it to be fully self-hosted in my lab.
• A rich editing environment.  Either a strong markdown editor or a rich text editor.  Something that supports HTML and other formats such as code, etc.
• The ability to create a group of related notes, notebook-esque, through nesting.
• Semantic searching.  I had to be able to easily search across all the notes for any key words or phrases and have the "distance" be reliable, in other words, have a good level of accuracy.
• AI insights through summaries of what I've written (again helpful for searching internally) but also be able to generate a strong prompt for searching the web.
• RAG.  Plain and simple, I want to be able to use my embeddings for RAG and converting that information to relevant articles and videos.
• Site scraping to create summaries and possible use it as captured references for my notes.

What are its components?

Synapse is comprised of six components which I will touch upon each component in detail in coming entries.

• .NET Aspire 
  • A Blazor website.  The Blazor website is my primary entry point for my notes.  It's written in C# (.NET 9).
  • One micro-service.  The service is a lightweight service that does ALLthe data interaction, returning only the needed data to the site or mobile app.
  • AppHost.  This is simply the .NET Aspire orchestrator,
• Ollama.  I have an Ollama server running in my homelab   This is used for generating the embeddings as well as my AI summaries and search prompts.
• SearXNG.  I'm self-hosting SerXNG to handle my searches across multiple sources (Google, Youtube, Stack Exchange, etc.).
• Firecrawl.  I'm using Firecrawl for my scraping tool.  This is self-hosted as well.
• FusionAuth.  I'm using FusionAuth for my authentication provider.
• Mobile App.  I created a .NET Maui/Blazor hybrid app to do nearly everything the website can do, creating a shared project to handle similar models, etc.

What challenges did I come across?

I will freely admit that I am a Microsoft fanboy when it comes to software development.  It's what I know, and it's what I fall back to nearly all the time.   That being said, I realized that good old MS has not been up to speed on some of this AI stuff when it comes to building it out and not relying on Copilot or something external.  When I looked at the infrastructure I'd need, vector databases were going to be biggest challenge.

PostGres was my first choice to go with, and setting up the database was easy, it understood vectors, the size (384, 768, etc.), and of course it generally has the performance to handle the searching and other AI-things needed.  I should note that I have been a fan of Dapper forever, and I love it as a small ORM, but it does not do well with AOT and doesn't run at all in a Docker container, so I decided to use EF Core this time, which presented another set of challenges, but at least it works within my chosen deployment environment.  Anyway, I was working with EF and PostGres, but never could get EF and PostGres to talk correctly, even after avoiding the code-first concept.   I spent a lot of time trying to the two of them to talk correctly.  In my frustration, I went back to my crutch and looked at SQL Server as my database.  I did some research and found out that SQL Server 2025 supports vectors as a field type now, even if it is in its early stages.  So, I decided to give it a try.  I'll go into more details about the setup later and the challenges with that move.

The next challenge was getting the performance good enough.  As you can guess, I am not running this on enterprise grade servers, it's a four-year-old Intel consumer processor with plenty of ram, but it's not really meant for hardcore server infrastructure, and I only have 1Gb networking in my tiny room.  I had to be smart about how I arranged the API service.  I moved everything I could into the API in order to keep the UI (both the web and mobile app) speedy.

Features of the entire system

I mentioned the core components earlier, but how does that translate into features that are useful?  Well, the concept of the system is based on notebooks, like NotebookLM, but instead of just being a research tool, it is also a personal knowledgebase.  Each notebook has notes associated within it.  The editor is a rich text editor that has your standard WYSIWYG features, including importing and importing Word docs and markdown docs.  This was always meant to be the primary feature of the system.  I added the AI component via both a summarization feature which can aid in semantic searching, and also an AI assistant that operates like ChatGPT or Gemini and can do general "assistance" or research.  The research functionality includes the note's content as part of it's context to help build out the RAG.   All of this creates a very powerful, locally hosted tool that can help me with my coding or enhancing my existing works.  I can also feed it my code and have it summarize the application for me.  There are few more features that I will discuss in future posts, but this outlines the main functionality.