OAQA / OpenQA Setup Guide

Anonymous (anonymous@undisclosed.example.com) — 2016-11-25T06:57:30+00:00

1390255308	current
Line 1:	Line 1:
	+	====== OAQA / OpenQA Setup Guide ======

	+	<note important>We consider this guide obsolete as we think we have a nicer software available now - see [[https://github.com/brmson/blanqa\|BlanQA]]!
	+
	+	BlanQA has nicer source code, is easier to clone and setup (no Indri JNI!), can run on top of a variety of text corpora (e.g. Wikipedia!), features //interactive question-answering mode// (and IRC gateway) and we are working to make it even better. Its algorithms are not as advanced as helloqa-prototype (below) yet, though.</note>
	+
	+	[[project:brmson:start\|{{ :project:brmson.png?220}}]]
	+
	+	This is a step-by-step setup guide for your very own IBM Watson-like software!
	+
	+	Here, we will set up a Linux-based, open-source question answering system that can process free text corpus and free-form English questions and answer them (with some low precision). Note that its performance and precision is not //anything// like DeepQA / IBM Watson obviously, but it's a start and it is built on the //same foundations// as the IBM project.
	+
	+	Note that the outcome is still an executable that is //not user-friendly at all//, it just spews a lot of cryptic output, not load wikipedia and start a nice conversation window. It answers questions based on batch files and you will need to wade through its debug output to figure out what's going on and what its answers are. It's just all very experimental at this point.
	+
	+	[[http://oaqa.github.io/\|OpenQA]] is a work mainly done at CMU (probably indirectly supported by IBM). It is being tweaked for user-friendlier setup by Pasky @ brmlab. OpenQA uses UIMA by IBM+Apache, Indri by the Lemur project, etc.
	+
	+	See [[project:brmson:start\|the brmson project page]] to learn more about the results of our research on state-of-art open source QA systems.
	+
	+	===== Installation =====
	+
	+	Anyway, to get this running on your (Debian Wheezy or some-such) machine, it should be enough to follow these steps, command by command:
	+
	+	* <code>apt-get install default-jdk maven uima-utils</code> to install the required toolchain
	+	* <code>git clone https://github.com/brmson/solr-provider; cd solr-provider; mvn install; cd ..</code> to install an up-to-date version of one boring OpenQA component
	+	* <code>git clone -b prototype https://github.com/brmson/helloqa helloqa-prototype</code> to get the current source of helloqa-prototype, already pre-cooked by brmsonners to be usable out-of-the-box
	+	* i?86 users (see your ''uname -a'') may skip this step, but x86_64/amd64 users need to compile their own indri database JNI bindings:
	+	* <code>apt-get install build-essential libz-dev</code>
	+	* <code>wget http://sourceforge.net/projects/lemur/files/lemur/indri-5.0/indri-5.0.tar.gz</code> the latest version is currently 5.6, but the indices in git were built with 5.0 and the version must match
	+	* <code>tar xf indri-5.0.tar.gz; cd indri-5.0; chmod +x configure</code>
	+	* <code>./configure --prefix= --enable-java --with-javahome=/usr/lib/jvm/default-java</code>
	+	* <code>make -j3</code>
	+	* <code>cp ./swig/obj/java/libindri_jni.so ../helloqa-prototype/lib/</code>
	+	* <code>cd ..</code>
	+	* <code>cd helloqa-prototype</code>
	+	* <code>mvn verify</code> to build it; the first time around, this will download several hundreds of megabytes of dependencies and chew for a while
	+	* <code>mvn exec:exec -Dexec.executable=java \
	+	-Dexec.args="-Djava.library.path=lib/ -classpath %classpath
	+	edu.cmu.lti.oaqa.ecd.driver.ECDDriver phases.err-analysis-IE-dsoqa"</code> to run it
	+	* In the massive debug prints, you will be able to fish for questions and proposed answers, congratulations! The test dataset is:
	+	* Freetext corpus ''src/main/resources/gs/dso-extension-psg.txt''
	+	* Questions ''src/main/resources/input/dso-extension.txt''
	+	* Right answers ''src/main/resources/gs/dso-extension-answerkey.txt''

Brmson

Anonymous (anonymous@undisclosed.example.com) — 2016-11-28T02:42:15+00:00

1480300725		current
Line 45:		Line 45:
	* [[http://www.aaai.org/Magazine/Watson/watson.php\|The AI Behind Watson]]; nice high-level overview scientific article		* [[http://www.aaai.org/Magazine/Watson/watson.php\|The AI Behind Watson]]; nice high-level overview scientific article
	* [[http://www.andrew.cmu.edu/user/ooo/watson/\|This is Watson! Special issue of the IBM research journal on Watson]]; extremely detailed and technical		* [[http://www.andrew.cmu.edu/user/ooo/watson/\|This is Watson! Special issue of the IBM research journal on Watson]]; extremely detailed and technical
-	* https://www.ibm.com/developerworks/community/blogs/InsideSystemStorage/entry/ibm_watson_how_to_build_your_own_watson_jr_in_your_basement7	+	* [[https://www.ibm.com/developerworks/community/blogs/InsideSystemStorage/entry/ibm_watson_how_to_build_your_own_watson_jr_in_your_basement7\|IBM Watson -- How to replicate Watson hardware and systems design for your own use in your basement]]
	* [[http://www.slideshare.net/jahendler/watson-summer-review82013final\|MiniDeepQA]]; technical summary of a student project replicating (in small scale) IBM Watson in cooperation with IBM Research!		* [[http://www.slideshare.net/jahendler/watson-summer-review82013final\|MiniDeepQA]]; technical summary of a student project replicating (in small scale) IBM Watson in cooperation with IBM Research!
	* [[http://debategraph.org/SolrSherlock\|SolrSherlock]]; another effort at recreating DeepQA, see also [[https://groups.google.com/forum/#!forum/qa-oss\|the qa-oss group]]; seems a bit unfocused and stalled to pasky, let's just watch them for now?		* [[http://debategraph.org/SolrSherlock\|SolrSherlock]]; another effort at recreating DeepQA, see also [[https://groups.google.com/forum/#!forum/qa-oss\|the qa-oss group]]; seems a bit unfocused and stalled to pasky, let's just watch them for now?

1480300725		current
Line 45:		Line 45:
	* [[http://www.aaai.org/Magazine/Watson/watson.php\|The AI Behind Watson]]; nice high-level overview scientific article		* [[http://www.aaai.org/Magazine/Watson/watson.php\|The AI Behind Watson]]; nice high-level overview scientific article
	* [[http://www.andrew.cmu.edu/user/ooo/watson/\|This is Watson! Special issue of the IBM research journal on Watson]]; extremely detailed and technical		* [[http://www.andrew.cmu.edu/user/ooo/watson/\|This is Watson! Special issue of the IBM research journal on Watson]]; extremely detailed and technical
-	* https://www.ibm.com/developerworks/community/blogs/InsideSystemStorage/entry/ibm_watson_how_to_build_your_own_watson_jr_in_your_basement7	+	* [[https://www.ibm.com/developerworks/community/blogs/InsideSystemStorage/entry/ibm_watson_how_to_build_your_own_watson_jr_in_your_basement7\|IBM Watson -- How to replicate Watson hardware and systems design for your own use in your basement]]
	* [[http://www.slideshare.net/jahendler/watson-summer-review82013final\|MiniDeepQA]]; technical summary of a student project replicating (in small scale) IBM Watson in cooperation with IBM Research!		* [[http://www.slideshare.net/jahendler/watson-summer-review82013final\|MiniDeepQA]]; technical summary of a student project replicating (in small scale) IBM Watson in cooperation with IBM Research!
	* [[http://debategraph.org/SolrSherlock\|SolrSherlock]]; another effort at recreating DeepQA, see also [[https://groups.google.com/forum/#!forum/qa-oss\|the qa-oss group]]; seems a bit unfocused and stalled to pasky, let's just watch them for now?		* [[http://debategraph.org/SolrSherlock\|SolrSherlock]]; another effort at recreating DeepQA, see also [[https://groups.google.com/forum/#!forum/qa-oss\|the qa-oss group]]; seems a bit unfocused and stalled to pasky, let's just watch them for now?

brmlab project:brmson

OAQA / OpenQA Setup Guide

Brmson