Indexing documents

The implementation of the Index method for the in-memory indexer is outlined as follows:

func (i *InMemoryBleveIndexer) Index(doc *index.Document) error {
    if doc.LinkID == uuid.Nil {
        return xerrors.Errorf("index: %w", index.ErrMissingLinkID)
    }
    doc.IndexedAt = time.Now()
    dcopy := copyDoc(doc)
    key := dcopy.LinkID.String()
    i.mu.Lock()
    if orig, exists := i.docs[key]; exists {
        dcopy.PageRank = orig.PageRank
    }
    if err := i.idx.Index(key, makeBleveDoc(dcopy)); err != nil {
        return xerrors.Errorf("index: %w", err)
    }
    i.docs[key] = dcopy
    i.mu.Unlock()
    return nil
}

To guarantee that the only way to mutate an already-indexed document is via a reindex operation, the indexer is designed to work with immutable copies of the documents that are passed as arguments to the Index method. The copyDoc helper creates a copy of the original document that we can safely store in the internal document map.

To add a new document to the index or to reindex an existing document, we need to provide bleve with two parameters: a string-based document ID and the document to be indexed. The makeBleveDoc helper returns a partial, lightweight view of the original document that, as we mentioned in the previous section, only contains the fields we want to use as part of our search queries.

When updating an existing document, we don't want the index operation to mutate the PageRank score that has already been assigned to the document as this would interfere with how the search results are ordered. To this end, if a document already exists, we need to patch the lightweight document that we pass to bleve so that it reflects the correct PageRank value.