How to solve this Error -> null: SimpleFSIndexInput(path="C:\Opengrok\data\index\project\_3e_Lucene50_0.tim")

[elkin5]

The error occurs when the suggester is not able to find a value

[The value in Tomcat execution]

03-Jul-2019 09:24:13.494 WARNING [suggester-lookup-78] org.opengrok.suggest.SuggesterSearcher.suggest Cannot perform suggester search
        java.io.IOException: null: SimpleFSIndexInput(path=""C:\Opengrok\data\index\project\_3e_Lucene50_0.doc")
                at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:188)
                at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
                at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
                at org.apache.lucene.codecs.lucene50.ForUtil.readBlock(ForUtil.java:187)
                at org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockDocsEnum.refillDocs(Lucene50PostingsReader.java:357)
                at org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockDocsEnum.nextDoc(Lucene50PostingsReader.java:382)
                at org.opengrok.suggest.SuggesterSearcher.getDocumentFrequency(SuggesterSearcher.java:315)
                at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:190)
                at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:117)
                at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:563)
                at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:523)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
        Caused by: java.nio.channels.ClosedChannelException
                at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
                at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:276)
                at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:50)
                at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:171)
                ... 14 more

[vladak]

What OpenGrok version are you running ?
How do you run the indexer exactly ?

[elkin5]

Last version.

Indexer Command:
java -Djava.util.logging.config.file=C:\Opengrok\opengrok-1.2.23\doc\logging.properties -jar C:\Opengrok\opengrok-1.2.23\lib\opengrok.jar -c C:\ctags\ctags.exe -s C:\Opengrok\src -d C:\Opengrok\data -H -P -S -G -W C:\Opengrok\config\configuration.xml

The archive C:\Opengrok\data\index\project_3e_Lucene50_0.doc exist in the directory

the search works, but when perform a search that does not have a suggester, the error comes out and does not work again the search

[vladak]

Does this happen for all projects or just for some ? Does this happen all the time ?

[elkin5]

when tomcat starts the search works on all the projects, when the error occurs, the search does not work for the project it was looking for, when it restarts tomcat it works again all the projects

[grajhn]

I get exactly the same error. I am using opengrok-1.2.7. As elkin5 mentioned above, if the suggested string is not found, it throws this exception on the console, and subsequent searches fail. We would have to bounce the Apache server to make this work again.

The current workaround that I have now is to disable suggester. I have created the read only configuration file and set the suggesterConfig property to false to disable suggestions.

[ahornace]

The error occurs when the suggester is not able to find a value

if the suggested string is not found, it throws this exception on the console

Could you please elaborate? I do not understand completely what the above statements mean.

Also, I do not see any connection how suggester could have hampered the search ability. Could you please post logs/errors that indicate search failures?

Furthermore, did it happen during reindex or at an arbitrary time?

Lastly, @grajhn are you running OpenGrok on Windows as well?

[vladak]

Given the file seems to be present on disk (there seems to be a discrepancy between issue title and the file reported in the stack trace - the first says _3e_Lucene50_0.tim, the second says _3e_Lucene50_0.doc) it looks as if someone closed the InputReader object used for initializing SuggesterSearcher.

[vladak]

As @ahornace said, it would be good to know when in the life cycle of the application does this happen.

[grajhn]

@vladak yes, I am running OpenGROK on Windows.
I was referring to Tomcat console that shows the exceptions. Here are some more details. This is not happening during indexing, but when searching for something for which there are no suggestions (or a matching string is not found).
For example: Assume that we are searching for a string "CDSTProvag", and that exact string is not there in the source code. This would result in an exception on the OpenGROK window with details as below.

Error
null: SimpleFSIndexInput(path="C:\opengrok-1.2.7\grokdata\index\CPP\_1d_Lucene50_0.tim")

Also, on the Tomcat console, this is what we see:
06-Aug-2019 15:51:32.669 WARNING [suggester-lookup-51] org.opengrok.suggest.SuggesterSearcher.suggest Cannot perform suggester search
 java.io.IOException: null: SimpleFSIndexInput(path="C:\opengrok-1.2.7\grokdata\index\CPP\_1d_Lucene50_0.tim")
        at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:188)
        at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
        at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
        at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125)
        at org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:221)
        at org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:154)
        at org.apache.lucene.codecs.blocktree.SegmentTermsEnum.next(SegmentTermsEnum.java:962)
        at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:203)
        at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:117)
        at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:563)
        at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:523)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:293)
        at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:50)
        at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:171)
06-Aug-2019 15:51:32.669 WARNING [suggester-lookup-52] org.opengrok.suggest.SuggesterSearcher.getComplexQueryData Could not get document ids for +full:cdstprovagent +full:. +full:*
 java.io.IOException: null: SimpleFSIndexInput(path="C:\opengrok-1.2.7\grokdata\
index\CPP\_1d_Lucene50_0.tim")
        at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:188)
        at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
        at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
        at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125)
        at org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:221)
        at org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:154)
        at org.apache.lucene.codecs.blocktree.SegmentTermsEnum.next(SegmentTermsEnum.java:962)
        at org.apache.lucene.search.MultiTermQueryConstantScoreWrapper$1.rewrite
(MultiTermQueryConstantScoreWrapper.java:175)
        at org.apache.lucene.search.MultiTermQueryConstantScoreWrapper$1.scorer(
MultiTermQueryConstantScoreWrapper.java:219)
        at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:143)
        at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorerSupplier(LRUQueryCache.java:751)
        at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.j
ava:364)
        at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:330)

        at org.apache.lucene.search.Weight.bulkScorer(Weight.java:177)
        at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:324)
        at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:832)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:667)

        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:471)

        at org.opengrok.suggest.SuggesterSearcher.getComplexQueryData(SuggesterSearcher.java:228)
        at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:155)
        at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:117)
        at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:563)
        at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:523)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedChannelException
        at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
        at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:276)
        at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:50)
        at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:171)
        ... 26 more
06-Aug-2019 15:51:32.684 WARNING [suggester-lookup-52] org.opengrok.suggest.SuggesterSearcher.suggest Cannot perform suggester search
 java.io.IOException: null: SimpleFSIndexInput(path="C:\opengrok-1.2.7\grokdata\
index\CPP\_1d_Lucene50_0.tim")
        at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:188)
        at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
        at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
        at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125)
        at org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:221)
        at org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:154)
        at org.apache.lucene.codecs.blocktree.SegmentTermsEnum.next(SegmentTermsEnum.java:894)
        at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:171)
        at org.opengrok.suggest.SuggesterSearcher.suggest(SuggesterSearcher.java:117)
        at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:563)
        at org.opengrok.suggest.Suggester$SuggesterSearchTask.call(Suggester.java:523)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedChannelException
        at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
        at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:276)
        at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:50)
        at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:171)
        ... 14 more

Once this error happens, the search stops working. We have to restart the Tomcat server.

[ahornace]

This seems like a Windows specific issue. I don't have any Windows machine available so I cannot reproduce. On Linux it works fine.

Also, you did not post any errors from the search. I do not immediately see how these errors could hamper the search. Quite the opposite, I would say that these errors are shown because search is not working.

I would advise upgrading to 1.3.0 which has upgraded Lucene which might solve the issue.

[vladak]

Okay, let me get this straight - (assuming the index is in place) once the Tomcat is started, the very first search for non-existing item in index leads to the observed behaviour ?

[elkin5]

Yes.

The procces for the replicate the problem(under condictions exposed above) is the next :

1. Index src folder data.

2. Opengrok works correctly.

3. Search for words that have no suggestions.

4. The error comes out.

5. Any search in opengrok not work. (Opengrok not works basically).

6. If restart tomcat the search works newly.