jruby/docs BETA
Lucene · intro

Index and search documents with Lucene

6 min read

Index and search documents with Lucene

Apache Lucene gives you millisecond full-text search without a search server.
Here is a self-contained script that fetches the Lucene jars, indexes a handful
of documents, then lets you query them interactively — copy, paste, run.

The code

require 'net/http'
require 'fileutils'

LUCENE_VERSION = '10.4.0'
JAR_DIR = File.join(__dir__, 'vendor', 'lucene')
JARS = %w[lucene-core lucene-analysis-common lucene-queryparser]

FileUtils.mkdir_p(JAR_DIR)
JARS.each do |name|
  dest = File.join(JAR_DIR, "#{name}-#{LUCENE_VERSION}.jar")
  next if File.exist?(dest)
  url = URI("https://repo1.maven.org/maven2/org/apache/lucene/#{name}/#{LUCENE_VERSION}/#{name}-#{LUCENE_VERSION}.jar")
  puts "Fetching #{name}..."
  File.binwrite(dest, Net::HTTP.get(url))
end
JARS.each { |name| require File.join(JAR_DIR, "#{name}-#{LUCENE_VERSION}.jar") }

require 'java'
java_import 'org.apache.lucene.store.ByteBuffersDirectory'
java_import 'org.apache.lucene.analysis.standard.StandardAnalyzer'
java_import 'org.apache.lucene.index.IndexWriter'
java_import 'org.apache.lucene.index.IndexWriterConfig'
java_import 'org.apache.lucene.document.Document'
java_import 'org.apache.lucene.document.TextField'
java_import 'org.apache.lucene.document.StringField'
java_import 'org.apache.lucene.document.Field'
java_import 'org.apache.lucene.index.DirectoryReader'
java_import 'org.apache.lucene.search.IndexSearcher'
java_import 'org.apache.lucene.queryparser.classic.QueryParser'

DOCS = [
  { id: '1', title: 'Getting started with JRuby',
    body: 'JRuby is a Ruby implementation on the JVM with access to the vast Java ecosystem.' },
  { id: '2', title: 'Full-text search with Lucene',
    body: 'Apache Lucene is a high-performance search library written in Java, powering Elasticsearch and Solr.' },
  { id: '3', title: 'SWT GUI programming in JRuby',
    body: 'SWT is the Eclipse widget toolkit. With JRuby you can build native desktop apps.' },
  { id: '4', title: 'Java interop in JRuby',
    body: 'JRuby makes Java interop simple: call methods, implement interfaces, and extend classes.' },
]

analyzer  = StandardAnalyzer.new
directory = ByteBuffersDirectory.new
writer    = IndexWriter.new(directory, IndexWriterConfig.new(analyzer))

DOCS.each do |d|
  doc = Document.new
  doc.add(StringField.new('id',    d[:id],    Field::Store::YES))
  doc.add(TextField.new('title',  d[:title], Field::Store::YES))
  doc.add(TextField.new('body',   d[:body],  Field::Store::NO))
  writer.add_document(doc)
end
writer.close

reader   = DirectoryReader.open(directory)
searcher = IndexSearcher.new(reader)
parser   = QueryParser.new('body', analyzer)

puts "#{DOCS.size} docs indexed. Enter a query (blank to quit):\n\n"

loop do
  print '> '
  input = $stdin.gets&.strip
  break if input.nil? || input.empty?

  begin
    hits = searcher.search(parser.parse(input), 10)
    if hits.totalHits.value == 0
      puts '  (no results)'
    else
      stored = searcher.storedFields()
      hits.scoreDocs.each do |sd|
        doc = stored.document(sd.doc)
        puts "  [#{doc.get('id')}] #{doc.get('title')} (score: #{sd.score.round(3)})"
      end
    end
  rescue => e
    puts "  Error: #{e.message}"
  end
  puts
end

reader.close

Running

ruby lucene-search.rb

The three Lucene jars are downloaded from Maven Central on first run and cached
in vendor/lucene/ next to the script. Subsequent runs are instant.

How it works

  • Net::HTTP.get pulls each JAR from Maven Central if not already cached — no Maven, no Gradle, no build tool needed
  • ByteBuffersDirectory is Lucene’s in-memory store; swap it for FSDirectory.open(Paths.get('my-index')) to persist the index to disk
  • StandardAnalyzer lowercases tokens, removes stop words, and applies stemming
  • TextField fields are tokenized and fully searchable; StringField fields are stored verbatim (IDs, slugs, enum values)
  • The writer must be closed before opening a DirectoryReader — Lucene requires the writer to flush before new documents become visible to readers
  • QueryParser understands Lucene query syntax: java AND search, title:JRuby, jvm~ (fuzzy match)

Next steps

  • Add a NumericDocValuesField for numeric sorting and range filters
  • Replace QueryParser with BooleanQuery.Builder to combine queries programmatically
  • Open a DirectoryReader from the writer (DirectoryReader.open(writer)) for near-real-time search without closing the writer
← All recipes