0.5 is not yet released and may contain bugs and regressions. However, it is shaping up nicely so I recommend you try it first. 0.4 remains available if you hit insurmountable problems; but please take time to file a ticket at github if you do.

Issue tracking at github.

Java 1.5 (or above) is required; the Sun version is recommended as it's regularly tested against.

1. Install Maven 2.
2. checkout repository
3. type 'mvn'
4. configure couchdb (see below)

You will now have a zip file in the target/ directory. This contains all the couchdb-lucene code, dependencies, startup scripts and configuration files to run couchdb-lucene.

[couchdb]
os_process_timeout=60000 ; increase the timeout from 5 seconds.

[external]
fti=/path/to/python /path/to/couchdb-lucene/tools/couchdb-external-hook.py

[httpd_db_handlers]
_fti = {couch_httpd_external, handle_external_req, <<"fti">>}

couchdb-lucene runs in a single, standalone JVM. As such, you can choose to locate your couchdb-lucene server on a different machine to couchdb if you wish, or keep it on the same machine, it's your call.

To start couchdb-lucene, run;

bin/run

To stop couchdb-lucene, simply kill the Java process.

You must supply a index function in order to enable couchdb-lucene as, by default, nothing will be indexed. To suppress a document from the index, return null. It's more typical to return a single Document object which contains everything you'd like to query and retrieve. You may also return an array of Document objects if you wish.

You may add any number of index views in any number of design documents. All searches will be constrained to documents emitted by the index functions.

Here's an complete example of a design document with couchdb-lucene features:

{
    "_id":"_design/a_design_document_with_any_name_you_like",
    "fulltext": {
        "by_subject": {
            "index":"function(doc) { var ret=new Document(); ret.add(doc.subject); return ret }"
        },
        "by_content": {
            "index":"function(doc) { var ret=new Document(); ret.add(doc.content); return ret }"
        }
    }
}

Here are some example URL's for the given design document;

https://localhost:5984/database/_fti/lucene/by_subject?q=hello
https://localhost:5984/database/_fti/lucene/by_content?q=hello

A fulltext object contains multiple index view declarations. An index view consists of;

analyzer

(optional) The analyzer to use

defaults

(optional) The default for numerous indexing options can be overridden here. A full list of options follows.

index

The indexing function itself, documented below.

The following indexing options can be defaulted;

Lucene has numerous ways of converting free-form text into tokens, these classes are called Analyzer's. By default, the StandardAnalyzer is used which lower-cases all text, drops common English words ("the", "and", and so on), among other things. This processing might not always suit you, so you can choose from several others by setting the "analyzer" field to one of the following values;

• brazilian
• chinese
• cjk
• czech
• dutch
• english
• french
• german
• keyword
• perfield
• porter
• russian
• simple
• standard
• thai

Note: You must also supply analyzer=<analyzer_name> as a query parameter to ensure that queries are processed correctly.

The "perfield" option lets you use a different analyzer for different fields and is configured as follows;

?analyzer=perfield:{field_name:"analyzer_name"}

Unless overridden, any field name not specified will be handled by the standard analyzer. To change the default, use the special default field name;

?analyzer=perfield:{default:"keyword"}

You should also specify the analyzer in your ddoc too;

"fulltext": {
  "idx": {
    "analyzer": "perfield:{default:\"keyword\"}"
  }
}

You may construct a new Document instance with;

var doc = new Document();

Data may be added to this document with the add method which takes an optional second object argument that can override any of the above default values.

// Add with all the defaults.
doc.add("value");

// Add a numeric field.
doc.add(35, {"type":"integer"});

// Add a date field.
doc.add(new Date("2009-01-01"), {"type":"date"});

// Add a date field (object must be a Date object

// Add a subject field.
doc.add("this is the subject line.", {"field":"subject"});

// Add but ensure it's stored.
doc.add("value", {"store":"yes"});

// Add but don't analyze.
doc.add("don't analyze me", {"index":"not_analyzed"});

// Extract text from the named attachment and index it (but not store it).
doc.attachment("attachment name", {"field":"attachments"});

function(doc) {
    var ret = new Document();

    function idx(obj) {
	for (var key in obj) {
	    switch (typeof obj[key]) {
	    case 'object':
		idx(obj[key]);
		break;
	    case 'function':
		break;
	    default:
		ret.add(obj[key]);
		break;
	    }
	}
    };

    idx(doc);

    if (doc._attachments) {
	for (var i in doc._attachments) {
	    ret.attachment("attachment", i);
	}
    }

    return ret;
}

function(doc) {
  return null;
}

function(doc) {
  var result = new Document();
  result.add(doc.subject, {"field":"subject", "store":"yes"});
  result.add(doc.content, {"field":"subject"});
  result.add(new Date(), {"field":"indexed_at"});
  return result;
}

function(doc) {
  var result = new Document();
  for(var a in doc._attachments) {
    result.attachment(a, {"field":"attachment"});
  }
  return result;
}

function(doc) {
    var mk = function(name, value, group) {
        var ret = new Document();
        ret.add(value, {"field": group, "store":"yes"});
        ret.add(group, {"field":"group", "store":"yes"});
        return ret;
    };
    if(doc.type != "reference") return null;
    var ret = new Array();
    for(var g in doc.groups) {
        ret.push(mk("library", doc.groups[g].library, g));
        ret.push(mk("method", doc.groups[g].method, g));
        ret.push(mk("target", doc.groups[g].target, g));
    }
    return ret;
}

Couchdb-lucene uses Apache Tika to index attachments of the following types, assuming the correct content_type is set in couchdb;

• Excel spreadsheets (application/vnd.ms-excel)
• HTML (text/html)
• Images (image/*)
• Java class files
• Java jar archives
• MP3 (audio/mp3)
• OpenDocument (application/vnd.oasis.opendocument.*)
• Outlook (application/vnd.ms-outlook)
• PDF (application/pdf)
• Plain text (text/plain)
• Powerpoint presentations (application/vnd.ms-powerpoint)
• RTF (application/rtf)
• Visio (application/vnd.visio)
• Word documents (application/msword)
• XML (application/xml)

You can perform all types of queries using Lucene's default query syntax.

In addition to normal text-based range searches (using the "field:[lower TO upper]" syntax), couchdb-lucene all supports numeric range searches for the following types: integer, long, float, double and date. The type is automatically derived from the search terms used, as follows;

Both the upper and lower bound must be of the same type to trigger numeric range searching. If they don't match, then a normal text-based range search is performed.

The following parameters can be passed for more sophisticated searches;

analyzer

The analyzer used to convert the query string into a query object.

callback

Specify a JSONP callback wrapper. The full JSON result will be prepended with this parameter and also placed with parentheses."

debug

Setting this to true disables response caching (the query is executed every time) and indents the JSON response for readability.

force_json

Usually couchdb-lucene determines the Content-Type of its response based on the presence of the Accept header. If Accept contains "application/json", you get "application/json" in the response, otherwise you get "text/plain;charset=utf8". Some tools, like JSONView for FireFox, do not send the Accept header but do render "application/json" responses if received. Setting force_json=true forces all response to "application/json" regardless of the Accept header.

include_docs

whether to include the source docs

limit

the maximum number of results to return

q

the query to run (e.g, subject:hello). If not specified, the default field is searched. Multiple q parameters are permitted, the resulting JSON will be an array of responses.

skip

the number of results to skip

sort

the comma-separated fields to sort on. Prefix with / for ascending order and \ for descending order (ascending is the default if not specified). Type-specific sorting is also available by appending a : and the sort type as normal (e.g, 'sort=amount:float'). Supported types are 'float', 'double', 'int', 'long' and 'date'.

stale=ok

If you set the stale option to ok, couchdb-lucene may not perform any refreshing on the index. Searches may be faster as Lucene caches important data (especially for sorting). A query without stale=ok will use the latest data committed to the index.

All parameters except 'q' are optional.

_id

The _id of the document.

All Dublin Core attributes are indexed and stored if detected in the attachment. Descriptions of the fields come from the Tika javadocs.

_dc.contributor

An entity responsible for making contributions to the content of the resource.

_dc.coverage

The extent or scope of the content of the resource.

_dc.creator

An entity primarily responsible for making the content of the resource.

_dc.date

A date associated with an event in the life cycle of the resource.

_dc.description

An account of the content of the resource.

_dc.format

Typically, Format may include the media-type or dimensions of the resource.

_dc.identifier

Recommended best practice is to identify the resource by means of a string or number conforming to a formal identification system.

_dc.language

A language of the intellectual content of the resource.

_dc.modified

Date on which the resource was changed.

_dc.publisher

An entity responsible for making the resource available.

_dc.relation

A reference to a related resource.

_dc.rights

Information about rights held in and over the resource.

_dc.source

A reference to a resource from which the present resource is derived.

_dc.subject

The topic of the content of the resource.

_dc.title

A name given to the resource.

_dc.type

The nature or genre of the content of the resource.

https://localhost:5984/dbname/_fti/design_doc/view_name?q=field_name:value
https://localhost:5984/dbname/_fti/design_doc/view_name?q=field_name:value&sort=other_field
https://localhost:5984/dbname/_fti/design_doc/view_name?debug=true&sort=billing_size&q=body:document AND customer:[A TO C]

The search result contains a number of fields at the top level, in addition to your search results.

etag

An opaque token that reflects the current version of the index. This value is also returned in an ETag header to facilitate HTTP caching.

fetch_duration

The number of milliseconds spent retrieving the documents.

limit

The maximum number of results that can appear.

q

The query that was executed.

rows

The search results array, described below.

search_duration

The number of milliseconds spent performing the search.

skip

The number of initial matches that was skipped.

total_rows

The total number of matches for this query.

The search results arrays consists of zero, one or more objects with the following fields;

doc

The original document from couch, if requested with include_docs=true

fields

All the fields that were stored with this match

id

The unique identifier for this match.

score

The normalized score (0.0-1.0, inclusive) for this match

Here's an example of a JSON response without sorting;

{
  "q": "+content:enron",
  "skip": 0,
  "limit": 2,
  "total_rows": 176852,
  "search_duration": 518,
  "fetch_duration": 4,
  "rows":   [
        {
      "id": "hain-m-all_documents-257.",
      "score": 1.601625680923462
    },
        {
      "id": "hain-m-notes_inbox-257.",
      "score": 1.601625680923462
    }
  ]
}

And the same with sorting;

{
  "q": "+content:enron",
  "skip": 0,
  "limit": 3,
  "total_rows": 176852,
  "search_duration": 660,
  "fetch_duration": 4,
  "sort_order":   [
        {
      "field": "source",
      "reverse": false,
      "type": "string"
    },
        {
      "reverse": false,
      "type": "doc"
    }
  ],
  "rows":   [
        {
      "id": "shankman-j-inbox-105.",
      "score": 0.6131107211112976,
      "sort_order":       [
        "enron",
        6
      ]
    },
        {
      "id": "shankman-j-inbox-8.",
      "score": 0.7492915391921997,
      "sort_order":       [
        "enron",
        7
      ]
    },
        {
      "id": "shankman-j-inbox-30.",
      "score": 0.507369875907898,
      "sort_order":       [
        "enron",
        8
      ]
    }
  ]
}

The Content-Type of the response is negotiated via the Accept request header like CouchDB itself. If the Accept header includes "application/json" then that is also the Content-Type of the response. If not, "text/plain;charset=utf-8" is used.

Calling couchdb-lucene without arguments returns a JSON object with information about the whole index.

https://127.0.0.1:5984/enron/_fti

returns;

{"doc_count":517350,"doc_del_count":1,"disk_size":318543045}