Hister – A private, full content search index that you control

文章洞見

原文內容目前無法安全取得，因此未產生文章摘要。

討論洞見

討論整體對 Hister 的核心概念與實用性反應熱烈：它建立個人化搜尋索引，涵蓋瀏覽頁面、書籤、瀏覽紀錄、本機檔案與網站爬取內容，並提供全文搜尋、semantic search、離線預覽、Web UI、CLI 與 MCP endpoint。使用者特別看重「找回曾經看過但未收藏的內容」、研究特定主題，以及自架與隱私特性；同時也提出行動端、匯入格式、檔案搜尋、儲存空間、語意搜尋處理方式與安全性等問題。

共識：多數留言正面評價 Hister 的個人知識檢索與研究用途，認為它比單純書籤、封存或瀏覽器歷史搜尋更符合需求；但對完整工作流程所需的整合功能與部分技術細節，仍有明顯未解問題。

實務建議

• Hister 最適合需要反覆查閱固定網站、研究資料集、個人瀏覽紀錄或本機筆記的人；可先從特定主題 corpus，而非全網爬取開始。
• 若部署在家庭或區域網路，應啟用 authentication，並注意 public mode 會讓任何人搜尋已索引內容；多使用者部署可利用 OIDC 與 per-user isolation。
• Android 使用者目前可考慮 Firefox extension；其他平台則需確認 Safari、iOS 與瀏覽器整合是否符合需求。
• 對重要頁面，現階段可把 content extraction 與 SingleFile/WARC 類完整封存視為不同需求，先確認 Hister 現有匯入與保存能力。
• 若使用 MCP 與 assistant integrations，建議限制可搜尋的 corpus 與帳號權限，避免將私人瀏覽內容暴露給不必要的服務。

待釐清問題

• Hister 是否已支援 SingleFile archive 的匯入、雙版本保存與回填？
• Readwise Reader、Zotero、瀏覽器 bookmarks，以及 Safari、iOS、Android 的完整匯入與分享流程何時或如何支援？
• semantic search 的 chunking、embedding、PDF parsing 與大型文件處理細節為何？
• 大規模 crawling 對 proxy、Cloudflare/CAPTCHA、更新排程、媒體與非 HTML 檔案的處理方式為何？
• 索引資料的實際儲存格式、磁碟使用量與長期成長率如何估算？
• browser extension 的權限範圍、資料傳輸方式與安全模型是否有更完整的說明？

代表性留言

1. Ohi, author here! Thanks for posting Hister. Feel free to A.M.A.My first free software search project was Searx, a privacy respecting metasearch engine, but because of the limitations of the metasearch concept, I've decided to take a different approach.Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites. It stores extracted content with offline result previews, so information remains searchable even when the original page changes or disappears. It supports full text and semantic search, can run entirely on your own machine, and includes a web interface, command line tools, and an MCP endpoint for assistant integrations.Project page: https://github.com/asciimoo/histerTiny read-only demo: https://demo.hister.org/

   asciimoo · HN #49402473

2. >Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites.Immediately interested and will check it out, thank you! I've wanted a "search stuff you've seen online" tool for a long time, but everything seems to be research-oriented or "archive but don't search" or some weird combination that means it's nigh useless to me. I've got decades of bookmarks and archives and I've kinda been stuck grepping them at best (it's rare but I do sometimes want a page I saw once three years ago and I love having that option), while hoping someone would build something better.One question if ya don't mind, while I explore: any chance of singlefile support? Content-extraction is useful in lots of situations (e.g. wallabag) and it's a great default, but sometimes it fails and sometimes you really do want the page, relatively close to how it actually was. Singlefile does that much better than most, and it does so well enough (and manually-handle-able enough if needed) that I don't feel any desire to switch to WARCs or similar.Though specifically I'm probably looking for something like "content-extract everything" + "key combo to save singlefile version too" + "upload singlefile archives to backfill / recover". Like 99% of the time content extraction is preferred, and I'm glad to see it... it's just not always enough, and having to go elsewhere for exceptions breaks a lot of the utility.

   Groxx · HN #49402752

3. Thanks you for making Hister. I have been using it since May and find it immensely useful. It's a piece of mind to know that I always will be able to easily find what I looked at. So I don't feel the need to bookmark things, which I used to do, but then never looked at anyways ;).

   human_llm · HN #49405988

4. I set this up a few months ago based on asciimoo's comments on HN, and barely used it at first, but I realized not too long ago that it could be a pretty useful research tool for one of my hobbies (award travel), that revolves around being in the know around various concepts and quirks.I scraped and imported posts from the blogs I regularly reference for award travel, then hooked it up to OpenCode/Codex as an MCP server and used that corpus for research on those topics. So I can ask things like "has anyone ever mentioned running into this problem before?" [1]If you have a hobby or working situation that requires you to regularly reference a core set of websites or reference materials, Hister provides almost all the tools out of the box to start a search engine against it. The default datasets they promote include the Python Stlib, MDN and RFC corpus, as an example. [2][1]: https://wmchen.com/blog/revisiting-hister/[2]: https://hister.org/datasets

   wmchen · HN #49402947

5. Wow, this is a really inspiring use case and blog post. Thanks for sharing it.What tools or features would Hister need to support your complete search workflow?

   asciimoo · HN #49403088

6. Oh this gets my brain spinning! Thanks for the tips

   Aboutplants · HN #49404107

7. I just finished setting this up yesterday and I'm kind of obsessed with it. I was previously a heavy user of Karakeep, but I hated forgetting to save something and losing it. I also think Hister's semantic search is a better solution than AI generated summaries and tags. Support for local docs is super cool too, I have it set up to index my org notes directory.

   quasigod · HN #49403309

8. While I love Karakeep, the truth is that I'm using it to solve this exact same problem, even though its project goals are not as well aligned. I would seriously consider giving Hister a try because it seems like a better fit. Bonus points for being written in Go.I suppose the one thing I need would be an easy way to share links into Hister. Hopefully we will see both Android/iOS mobile apps to send links into Hister and better support for Safari. I wonder what could be possible with the extensions support available on mobile browsers too (Safari, MS Edge, Firefox).

   evanjrowley · HN #49409407

9. the Hister extension is available on Firefox for Android :)https://addons.mozilla.org/en-US/firefox/addon/hister/

   bpavuk · HN #49410507

10. - lots of stupid questions to the author from a guy who has no idea about search engines- let us say I want to index every blog ever listed on HN- should be a small subset of the 400 billion pages out there on the internet no?- First I need to gather data, what do you use to load so many webpages rapidly? asyncio with aiohttp in python? are there better options?- how do you handle proxies? rotation? are there libraries you recommend for this?- what about pages that use cloudflare? or block your request or present a captcha or a challenge of some kind?- what are the filetypes you collect? only html or media as well?- where and in what format do you store all these collected files? flat file storage? duckdb? postgres? hstore? something else?- what is the frequency at which you refresh each page? once a day? once a week? something else?- what kind of pre-processing do you use on the collected data? remove extra spaces? special characters? some kind of complex regex pipeline? LLM?- how do you match the incoming query with processed data? simple text matching? regex? vector embedding match? something else?

    vivzkestrel · HN #49406404

11. I have created something similar, yet different. I maintain my own Internet domain index.https://github.com/rumca-js/Internet-Places-DatabaseAlso I maintain android app that can be used to search places.https://f-droid.org/en/packages/io.github.rumcajs.offlineweb...I believe hister you have to fill in with your data, right?

    renegat0x0 · HN #49402863

12. I tried this out this week and liked it but really wish this project had some form of auth. Opening the contents of every page you’ve ever visited, even to the local network, is not the best idea.

    Carrok · HN #49402653

13. Hister supports token based, password based, and OIDC/OAuth authentications with optional multi-user handling. Details about user handling can be found here: https://hister.org/docs/user-handlingIt also has a "public mode" where anyone can search the indexed content, but only authenticated users can add or modify it.

    asciimoo · HN #49402811

14. I've set it up today with OIDC AuthN (Keycloak), removed the username/password fields & kept only OIDC, it's working flawless.Also indexed data is persisted on a per-user basis, so you got this isolation and certainty that your searches will not be polluted by your family's

    androidu · HN #49403219

15. This is so cool! For the time being I’m still locked into notion for my handwritten knowledge base, but I love this for incorporating external information.For the semantic search is there any chunking/processing that happens with the content or do you need to be diligent about having a large embedding context (and/or small content)?

    mmargenot · HN #49405837

16. Any chance I could integrate or import data from Readwise Reader? (Link archival tool)

    retrochameleon · HN #49410825

17. Love to see this. I started something similar a couple years ago but didn't quite have the patience to make the browser extensions really bulletproof.Planning to contribute significant improvements to the vector search side here.

    jbellis · HN #49405384

18. It’s always been bizarre to me how bad browsers themselves are at searching their own history. This looks great.

    dmd · HN #49407911

19. Would be really cool to have Zotero library integration / import support!

    cowmanOG · HN #49404495

20. What's the disk space usage like? i.e. average per-day/week additional storage in your usage?

    ndom91 · HN #49407610

21. Ooh I’ve been thinking about this idea for years, I’m glad someone beat me to it. Guess I have something to play with over the rest of the weekend!

    robotmay · HN #49402842

22. I LOVE the concept. I will play around with the execution, if it works as described this is a great product.

    lnenad · HN #49403175

23. Thanks, this will pair wonderfully with my Capcat.org project.

    stayux · HN #49406921

24. Ister (Latin Hister, Ancient Greek Ἴστρος / Istros) is the ancient classical name used by the Greeks and Romans for the Danube River, specifically referring to its lower course.

    phendrenad2 · HN #49408169

25. Is the design vibe coded in Codex by any chance? Seeing a lot of similar designs out there and it’s really starting to annoy me.

    ebr4him · HN #49403866

26. The whole design is created by a main contributor and IIRC the style is called neobrutalism.

    asciimoo · HN #49403921

27. I think that the tells, if they are in fact legitimate, are negligible at best or subtle at worst. It looks like the site uses Tailwind, so if AI was involved it wasn't generating the design from scratch. If it did then I'm impressed because usually it sucks. This is alright.

    tolerance · HN #49405745

28. can it search .pdf content as easy as searching content in txt file?

    triilman · HN #49405917

29. Any way to add for mobile?

    kinibha · HN #49404740

30. If you're on android the extension works for Firefox

    quasigod · HN #49405142

31. Time to shameless plug my own easy to host your own content search indexWebtm.ioAll open source and small enough to deploy. I deploy to cf webworkers so it’s the only place it’s tested.One cool thing is we work on iOS, chrome and friends, Firefox and pretty much everywhere. We do require you bring your own LLM though.

    alexnewman · HN #49403231

32. Cant wait to check this out

    0n0n0m0uz · HN #49406117

33. Can it import browser bookmarks? It says browser history and a bunch of other bookmark services but not specifically browser bookmarks.

    exe34 · HN #49403178

34. Yes, it can import data from various sources, including browser history, and popular self-hosted bookmarking apps: https://hister.org/docs/import

    asciimoo · HN #49403199

35. I love the idea overall, but something about browser extensions give me a bad taste. Am I overthinking it? I realize the extension is open source, of course.

    knowaveragejoe · HN #49404729

36. This is like Hipster. But missing a 'p'.

    shevy-java · HN #49402784