Basic tool and API to check the health of an index and write a new segments file that removes reference to problematic segments.
As this tool checks every byte in the index, on a large index it can take quite a long time to run.
Create a new CheckIndex on the directory.
<init>(org.apache.lucene.store.Directory dir, org.apache.lucene.store.Lock writeLock)Expert: create a directory with the specified lock. This should really not be used except for unit tests!!!! It exists only to support special tests (such as TestIndexWriterExceptions*), that would otherwise be more complicated to debug if they had to close the writer for each check.
assertsOn()Check whether asserts are enabled or not.
Returns: true iff asserts are enabled
main(String[] args)Command-line interface to check and exorcise corrupt segments from an index.
Run it like this:
java -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex pathToIndex [-exorcise] [-verbose] [-segment X] [-segment Y]
-exorcise: actually write a new segments_N file, removing any problematic
segments. *LOSES DATA*
-segment X: only check the specified segment(s). This can be specified
multiple times, to check more than one segment: -segment _2 * -segment _a.
You can't use this with the -exorcise option.
WARNING: -exorcise should only be used on an emergency basis as it will
cause documents (perhaps many) to be permanently removed from the index. Always make a backup
copy of your index before running this! Do not run this tool on an index that is actively being
written to. You have been warned!
Run without -exorcise, this tool will open the index, report version information and report any exceptions it hits and what action it would take if -exorcise were specified. With -exorcise, this tool will remove any segments that have issues and write a new segments_N file. This means all documents contained in the affected segments will be removed.
This tool exits with exit code 1 if the index cannot be opened or has any corruption, else 0.
Parse command line args into fields
| name | type | description |
|---|---|---|
| args | String[] | The command line arguments |
Returns: An Options struct
| IllegalArgumentException | if any of the CLI args are invalid | |
testDocValues(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean failFast)Test docvalues.
testFieldInfos(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean failFast)Test field infos.
testFieldNorms(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean failFast)Test field norms.
testLiveDocs(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean failFast)Test live docs.
testPoints(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean failFast)Test the points index.
testPostings(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream)Test the term index.
testPostings(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean verbose, int level, boolean failFast)Test the term index.
testSort(org.apache.lucene.index.CodecReader reader, org.apache.lucene.search.Sort sort, java.io.PrintStream infoStream, boolean failFast)Tests index sort order.
testStoredFields(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean failFast)Test stored fields.
testTermVectors(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream)Test term vectors.
testTermVectors(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean verbose, int level, boolean failFast)Test term vectors.
testVectors(org.apache.lucene.index.CodecReader reader, java.io.PrintStream infoStream, boolean failFast)Test the vectors index.
Returns a Status instance detailing the state of the index.
As this method checks every byte in the index, on a large index it can take quite a long time to run.
WARNING: make sure you only call this when the index is not opened by any writer.
checkIndex(List<String> onlySegments)Returns a Status instance detailing the state of the index.
| name | type | description |
|---|---|---|
| only_segments | List<String> | list of specific segment names to check
As this method checks every byte in the specified segments, on a large index it can take quite a long time to run. |
checkIndex(List<String> onlySegments, java.util.concurrent.ExecutorService executorService)Returns a Status instance detailing the state of the index.
This method allows caller to pass in customized ExecutorService to speed up the check.
WARNING: make sure you only call this when the index is not opened by any writer.
close()doCheck(org.apache.lucene.index.CheckIndex.Options opts)Actually perform the index check
| name | type | description |
|---|---|---|
| opts | org.apache.lucene.index.CheckIndex.Options | The options to use for this check |
Returns: 0 iff the index is clean, 1 otherwise
exorciseIndex(org.apache.lucene.index.CheckIndex.Status result)Repairs the index using previously returned result from #checkIndex. Note that this
does not remove any of the unreferenced files after it's done; you must separately open an
IndexWriter, which deletes unreferenced files when it's created.
WARNING: this writes a new segments file into the index, effectively removing all documents in broken segments from the index. BE CAREFUL.
getFailFast() · also: get_fail_fastSee #setFailFast.
setFailFast(boolean v) · also: set_fail_fastIf true, just throw the original exception immediately when corruption is detected, rather than continuing to iterate to other segments looking for more corruption.
setInfoStream(java.io.PrintStream out) · also: set_info_streamSet infoStream where messages should go. See #setInfoStream(PrintStream,boolean).
getLevel() · also: get_levelSee #setLevel.
setLevel(int v) · also: set_levelSets Level, the higher the value, the more additional checks are performed. This will likely
drastically increase time it takes to run CheckIndex! See Level
setInfoStream(java.io.PrintStream out, boolean verbose)Set infoStream where messages should go. If null, no messages are printed. If verbose is true then more details are printed.
setThreadCount(int tc) · also: set_thread_countSet threadCount used for parallelizing index integrity checking.