Error if fulltext search phrase contains ']'

[andreas@…]

[jira2trac import : issue created on September 14, 2006 10:44:35 AM CEST ​http://issues.cocoondev.org/browse/DSY-347 ]

In the fulltext search of the wiki, search for a phrase like paramslocaleAsString?'

An error is thrown:

Received exception from repository server.
Error performing query.
Error parsing query.
Encountered "]" at line 1, column 23. Was expecting one of: "TO" ... <RANGEIN_QUOTED> ... <RANGEIN_GOOP> ...

Cocoon Stack Trace
Message: Sitemap: error calling function 'org.outerj.daisy.frontend.FulltextSearchApple?'
context://daisy/sitemap.xmap - 727:78 <map:call>

Java Stack Trace
Message: Sitemap: error calling function 'org.outerj.daisy.frontend.FulltextSearchApple?'
Class: org.apache.cocoon.ProcessingException?
Stacktrace:
org.apache.cocoon.ProcessingException?.throwLocated (ProcessingException?.java:112)
org.apache.cocoon.components.treeprocessor.sitemap.CallFunctionNode?.invoke (CallFunctionNode?.java:140)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:46)
org.apache.cocoon.components.treeprocessor.sitemap.PreparableMatchNode?.invoke (PreparableMatchNode?.java:130)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:46)
org.apache.cocoon.components.treeprocessor.sitemap.PreparableMatchNode?.invoke (PreparableMatchNode?.java:130)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode?.invoke (PipelineNode?.java:142)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode?.invoke (PipelinesNode?.java:92)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:234)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:176)
org.apache.cocoon.components.treeprocessor.TreeProcessor?.process (TreeProcessor?.java:252)
org.apache.cocoon.components.treeprocessor.sitemap.MountNode?.invoke (MountNode?.java:117)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:46)
org.apache.cocoon.components.treeprocessor.sitemap.PreparableMatchNode?.invoke (PreparableMatchNode?.java:130)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode?.invoke (PipelineNode?.java:142)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode?.invoke (PipelinesNode?.java:92)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:234)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:176)
org.apache.cocoon.components.treeprocessor.TreeProcessor?.process (TreeProcessor?.java:252)
org.apache.cocoon.Cocoon.process (Cocoon.java:686)
org.apache.cocoon.servlet.CocoonServlet?.service (CocoonServlet?.java:1153)
javax.servlet.http.HttpServlet?.service (HttpServlet?.java:802)
org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter (ApplicationFilterChain?.java:252)
org.apache.catalina.core.ApplicationFilterChain?.doFilter (ApplicationFilterChain?.java:173)
org.outerj.daisy.frontend.util.CharacterEncodingFilter?.doFilter (CharacterEncodingFilter?.java:35)
org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter (ApplicationFilterChain?.java:202)
org.apache.catalina.core.ApplicationFilterChain?.doFilter (ApplicationFilterChain?.java:173)
org.apache.catalina.core.StandardWrapperValve?.invoke (StandardWrapperValve?.java:213)
org.apache.catalina.core.StandardContextValve?.invoke (StandardContextValve?.java:178)
org.apache.catalina.core.StandardHostValve?.invoke (StandardHostValve?.java:126)
org.apache.catalina.valves.ErrorReportValve?.invoke (ErrorReportValve?.java:105)
org.apache.catalina.core.StandardEngineValve?.invoke (StandardEngineValve?.java:107)
org.apache.catalina.connector.CoyoteAdapter?.service (CoyoteAdapter?.java:148)
org.apache.jk.server.JkCoyoteHandler?.invoke (JkCoyoteHandler?.java:199)
org.apache.jk.common.HandlerRequest?.invoke (HandlerRequest?.java:282)
org.apache.jk.common.ChannelSocket?.invoke (ChannelSocket?.java:754)
org.apache.jk.common.ChannelSocket?.processConnection (ChannelSocket?.java:684)
org.apache.jk.common.ChannelSocket?$SocketConnection?.runIt (ChannelSocket?.java:876)
org.apache.tomcat.util.threads.ThreadPool?$ControlRunnable?.run (ThreadPool?.java:684)
java.lang.Thread.run (Thread.java:595)
Message: Received exception from repository server.
Class: org.outerj.daisy.repository.RepositoryException?
Stacktrace:
org.outerj.daisy.repository.clientimpl.infrastructure.DaisyHttpClient?.handleNotOkResponse (DaisyHttpClient?.java:155)
org.outerj.daisy.repository.clientimpl.infrastructure.DaisyHttpClient?.executeMethod (DaisyHttpClient?.java:86)
org.outerj.daisy.repository.clientimpl.query.RemoteQueryManager?.performQuery (RemoteQueryManager?.java:105)
org.outerj.daisy.frontend.FulltextSearchApple?.processInternal (FulltextSearchApple?.java:105)
org.outerj.daisy.frontend.util.AbstractDaisyApple?.process (AbstractDaisyApple?.java:55)
org.apache.cocoon.components.flow.apples.ApplesProcessor?.processApple (ApplesProcessor?.java:138)
org.apache.cocoon.components.flow.apples.ApplesProcessor?.callFunction (ApplesProcessor?.java:81)
org.apache.cocoon.components.treeprocessor.sitemap.CallFunctionNode?.invoke (CallFunctionNode?.java:138)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:46)
org.apache.cocoon.components.treeprocessor.sitemap.PreparableMatchNode?.invoke (PreparableMatchNode?.java:130)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:46)
org.apache.cocoon.components.treeprocessor.sitemap.PreparableMatchNode?.invoke (PreparableMatchNode?.java:130)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode?.invoke (PipelineNode?.java:142)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode?.invoke (PipelinesNode?.java:92)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:234)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:176)
org.apache.cocoon.components.treeprocessor.TreeProcessor?.process (TreeProcessor?.java:252)
org.apache.cocoon.components.treeprocessor.sitemap.MountNode?.invoke (MountNode?.java:117)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:46)
org.apache.cocoon.components.treeprocessor.sitemap.PreparableMatchNode?.invoke (PreparableMatchNode?.java:130)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode?.invoke (PipelineNode?.java:142)
org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode?.invokeNodes (AbstractParentProcessingNode?.java:68)
org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode?.invoke (PipelinesNode?.java:92)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:234)
org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor?.process (ConcreteTreeProcessor?.java:176)
org.apache.cocoon.components.treeprocessor.TreeProcessor?.process (TreeProcessor?.java:252)
org.apache.cocoon.Cocoon.process (Cocoon.java:686)
org.apache.cocoon.servlet.CocoonServlet?.service (CocoonServlet?.java:1153)
javax.servlet.http.HttpServlet?.service (HttpServlet?.java:802)
org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter (ApplicationFilterChain?.java:252)
org.apache.catalina.core.ApplicationFilterChain?.doFilter (ApplicationFilterChain?.java:173)
org.outerj.daisy.frontend.util.CharacterEncodingFilter?.doFilter (CharacterEncodingFilter?.java:35)
org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter (ApplicationFilterChain?.java:202)
org.apache.catalina.core.ApplicationFilterChain?.doFilter (ApplicationFilterChain?.java:173)
org.apache.catalina.core.StandardWrapperValve?.invoke (StandardWrapperValve?.java:213)
org.apache.catalina.core.StandardContextValve?.invoke (StandardContextValve?.java:178)
org.apache.catalina.core.StandardHostValve?.invoke (StandardHostValve?.java:126)
org.apache.catalina.valves.ErrorReportValve?.invoke (ErrorReportValve?.java:105)
org.apache.catalina.core.StandardEngineValve?.invoke (StandardEngineValve?.java:107)
org.apache.catalina.connector.CoyoteAdapter?.service (CoyoteAdapter?.java:148)
org.apache.jk.server.JkCoyoteHandler?.invoke (JkCoyoteHandler?.java:199)
org.apache.jk.common.HandlerRequest?.invoke (HandlerRequest?.java:282)
org.apache.jk.common.ChannelSocket?.invoke (ChannelSocket?.java:754)
org.apache.jk.common.ChannelSocket?.processConnection (ChannelSocket?.java:684)
org.apache.jk.common.ChannelSocket?$SocketConnection?.runIt (ChannelSocket?.java:876)
org.apache.tomcat.util.threads.ThreadPool?$ControlRunnable?.run (ThreadPool?.java:684)
java.lang.Thread.run (Thread.java:595)
Message: Error performing query.
Class: org.outerj.daisy.repository.query.QueryException?
Stacktrace:
ATTENTION: this is the stacktrace of an exception that happened on the repository server.
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQueryReturnDocuments (LocalQueryManager?.java:550)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQuery (LocalQueryManager?.java:112)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQuery (LocalQueryManager?.java:101)
org.outerj.daisy.httpconnector.handlers.QueryHandler?.handleRequest (QueryHandler?.java:68)
org.outerj.daisy.httpconnector.HttpConnectorImpl?$PathHandler?.handle (HttpConnectorImpl?.java:244)
org.outerj.daisy.httpconnector.HttpConnectorImpl?$DaisyHandler?.handle (HttpConnectorImpl?.java:203)
org.mortbay.http.HttpContext?.handle (HttpContext?.java:1530)
org.mortbay.http.HttpContext?.handle (HttpContext?.java:1482)
org.mortbay.http.HttpServer?.service (HttpServer?.java:909)
org.mortbay.http.HttpConnection?.service (HttpConnection?.java:816)
org.mortbay.http.HttpConnection?.handleNext (HttpConnection?.java:982)
org.mortbay.http.HttpConnection?.handle (HttpConnection?.java:833)
org.mortbay.http.SocketListener?.handleConnection (SocketListener?.java:244)
org.mortbay.util.ThreadedServer?.handle (ThreadedServer?.java:357)
org.mortbay.util.ThreadPool?$PoolThread?.run (ThreadPool?.java:534)
Message: Error parsing query.
Class: org.outerj.daisy.repository.query.QueryException?
Stacktrace:
ATTENTION: this is the stacktrace of an exception that happened on the repository server.
org.outerj.daisy.ftindex.FullTextIndexImpl?.search (FullTextIndexImpl?.java:156)
sun.reflect.NativeMethodAccessorImpl?.invoke0 [native method] (NativeMethodAccessorImpl?.java:-2)
sun.reflect.NativeMethodAccessorImpl?.invoke (NativeMethodAccessorImpl?.java:39)
sun.reflect.DelegatingMethodAccessorImpl?.invoke (DelegatingMethodAccessorImpl?.java:25)
java.lang.reflect.Method.invoke (Method.java:585)
org.apache.avalon.activation.impl.ApplianceInvocationHandler?.invoke (ApplianceInvocationHandler?.java:129)
$Proxy12.search (:-1)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQueryReturnDocuments (LocalQueryManager?.java:395)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQuery (LocalQueryManager?.java:112)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQuery (LocalQueryManager?.java:101)
org.outerj.daisy.httpconnector.handlers.QueryHandler?.handleRequest (QueryHandler?.java:68)
org.outerj.daisy.httpconnector.HttpConnectorImpl?$PathHandler?.handle (HttpConnectorImpl?.java:244)
org.outerj.daisy.httpconnector.HttpConnectorImpl?$DaisyHandler?.handle (HttpConnectorImpl?.java:203)
org.mortbay.http.HttpContext?.handle (HttpContext?.java:1530)
org.mortbay.http.HttpContext?.handle (HttpContext?.java:1482)
org.mortbay.http.HttpServer?.service (HttpServer?.java:909)
org.mortbay.http.HttpConnection?.service (HttpConnection?.java:816)
org.mortbay.http.HttpConnection?.handleNext (HttpConnection?.java:982)
org.mortbay.http.HttpConnection?.handle (HttpConnection?.java:833)
org.mortbay.http.SocketListener?.handleConnection (SocketListener?.java:244)
org.mortbay.util.ThreadedServer?.handle (ThreadedServer?.java:357)
org.mortbay.util.ThreadPool?$PoolThread?.run (ThreadPool?.java:534)
Message: Encountered "]" at line 1, column 23. Was expecting one of: "TO" ... <RANGEIN_QUOTED> ... <RANGEIN_GOOP> ...
Class: org.apache.lucene.queryParser.ParseException?
Stacktrace:
ATTENTION: this is the stacktrace of an exception that happened on the repository server.
org.apache.lucene.queryParser.QueryParser?.generateParseException (QueryParser?.java:1226)
org.apache.lucene.queryParser.QueryParser?.jj_consume_token (QueryParser?.java:1109)
org.apache.lucene.queryParser.QueryParser?.Term (QueryParser?.java:883)
org.apache.lucene.queryParser.QueryParser?.Clause (QueryParser?.java:741)
org.apache.lucene.queryParser.QueryParser?.Query (QueryParser?.java:712)
org.apache.lucene.queryParser.QueryParser?.parse (QueryParser?.java:122)
org.apache.lucene.queryParser.MultiFieldQueryParser?.parse (MultiFieldQueryParser?.java:171)
org.outerj.daisy.ftindex.FullTextIndexImpl?.search (FullTextIndexImpl?.java:144)
sun.reflect.NativeMethodAccessorImpl?.invoke0 [native method] (NativeMethodAccessorImpl?.java:-2)
sun.reflect.NativeMethodAccessorImpl?.invoke (NativeMethodAccessorImpl?.java:39)
sun.reflect.DelegatingMethodAccessorImpl?.invoke (DelegatingMethodAccessorImpl?.java:25)
java.lang.reflect.Method.invoke (Method.java:585)
org.apache.avalon.activation.impl.ApplianceInvocationHandler?.invoke (ApplianceInvocationHandler?.java:129)
$Proxy12.search (:-1)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQueryReturnDocuments (LocalQueryManager?.java:395)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQuery (LocalQueryManager?.java:112)
org.outerj.daisy.repository.serverimpl.query.LocalQueryManager?.performQuery (LocalQueryManager?.java:101)
org.outerj.daisy.httpconnector.handlers.QueryHandler?.handleRequest (QueryHandler?.java:68)
org.outerj.daisy.httpconnector.HttpConnectorImpl?$PathHandler?.handle (HttpConnectorImpl?.java:244)
org.outerj.daisy.httpconnector.HttpConnectorImpl?$DaisyHandler?.handle (HttpConnectorImpl?.java:203)
org.mortbay.http.HttpContext?.handle (HttpContext?.java:1530)
org.mortbay.http.HttpContext?.handle (HttpContext?.java:1482)
org.mortbay.http.HttpServer?.service (HttpServer?.java:909)
org.mortbay.http.HttpConnection?.service (HttpConnection?.java:816)
org.mortbay.http.HttpConnection?.handleNext (HttpConnection?.java:982)
org.mortbay.http.HttpConnection?.handle (HttpConnection?.java:833)
org.mortbay.http.SocketListener?.handleConnection (SocketListener?.java:244)
org.mortbay.util.ThreadedServer?.handle (ThreadedServer?.java:357)
org.mortbay.util.ThreadPool?$PoolThread?.run (ThreadPool?.java:534)

[paul]

fulltext.diff

[paul]

[jira2trac import : comment created by bruno on September 15, 2006 3:54:18 AM CEST]

This is to be expected, as what you enter in the fulltext box is not just a string but a Lucene query.

I do agree this is not so nice, ideally we should have a never-fails query syntax for the fulltext searches. Maybe something like this exists already in Lucene.

[paul]

[jira2trac import : comment created by deining on September 15, 2006 4:20:53 AM CEST]

Isn't that a matter of simply escaping special characters prior to submitting the search phrase to the search engine?
See: ​http://lucene.apache.org/java/docs/queryparsersyntax.html

If I'm searching for something like: row\[d\:value\[1\] = $currentDate\] on ​http://cocoondev.org/daisy/search, I'm getting two results.

[paul]

[jira2trac import : comment created by bruno on September 15, 2006 4:27:52 AM CEST]

Yes and no, as I don't want to disable the ability to use the query syntax.

The comprehensive listing of special characters in the lucene docs makes the escaping simpler than I had imagined at first.

Maybe we should default to escaping special characters when using the fulltext search box, and provide a checkbox on the fulltext search page to enable/disable this for when one wants to use the special characters.

[paul]

[jira2trac import : comment created by bruno on September 15, 2006 1:47:43 PM CEST]

BTW, patch welcome.

I would suggest adding a method QueryHelper?.escapeFullTextQuery(...) which escapes the special characters.

The rest of the changes would be in the class FulltextSearchApple? and fulltextsearch_definition.xml / fulltextsearch_template.xml

[paul]

[jira2trac import : comment created by bruno on September 16, 2006 10:47:51 AM CEST]

Applied.

I've changed the implementation of the escapeFullTextQuery method a bit, so that it is more efficient (though efficiency is not very important for this) and imo a bit more readable.

I've noticed you've left out the ~ character -- I'm assuming this wasn't on purpose, let me know if otherwise.