Synology Lucene++ Client

Dev

Recently I was trying to figure out a way of accessing the “universal search” index available via the Synology UI, in a way that didn’t require logging into the UI itself.

tl;dr here are the repos:

Synology Lucene Client REST backend (GitHub.com)

Synology Lucene Client frontend (GitHub.com)

Requires Docker (ie container service) to be installed on the Synology NAS!

quick background

I wanted to be able to access the index service directly and could see that the NAS had a synoelasticd service running, but couldn’t see any traffic across the socket that looked like any normal conversation with Elastic. Of course, there’s a Synology REST API, but this is inflexible in terms of how it permits access to the “ES” (Lucene) backend. Off I went to dig around the box.

I discovered that the indexing service creates a Lucene index, under /<volume>/<share>/@eaDir/SYNO@.fileindexdb. This index is compatible with Lucene++ ~v3.0.9, or Lucene (for Java) ~v4.x. In fact, you can load the indices via Luke 4.x, as shown below:

Screenshot of Luke 4.x loading Lucene++ indices created by the Synology Universal Search tool

I suppose the objectives were as follows:

  • Allow me to send queries to a REST interface
  • Return search results (with context highlights)
  • Allow me to pull back the document itself

With that in mind I’ve written a REST backend that accesses the Lucene++ indices in a read-only fashion; running in Docker and mounting only the directories we want to be able to access.

It’s accompanied by a proof-of-concept UI that allows the documents themselves to be previewed in the same browser window:

Screenshot of UI for pulling back documents from SynoSearch REST UI

This is an alpha release: there are known issues re eg path traversal (eg I can pull back /get?p=%2Fetc%2Fshadow) so it’s not meant to be deployed in anything like a hostile environment. No checks are performed re user access so if you’re mounting a volume with varying permissions across folders (eg /homes) then all file contents can be queried by all users.

Todo list:

  • Preview documents other than PDF (eg MSO, images, etc)
  • Cache thumbnails / smaller versions somewhere for quick previews
  • User authorisation and filtering results based on current user (eg is there a way of creating an access token within the Synology UI that will allow us to resolve a request to a particular UID / GID? In which case, the permissions within the Lucene index can be used…)
  • Tighten up path traversal based on user authorisation
  • User-friendlier config of the runtime options (ie where we’re specifying indices to mount)
  • Add logic to search across more than one index

Pull requests welcome!