Changes/Elasticsearch

Elasticsearch

Summary

Goal of this change is to pack Elasticsearch into main fedora repo.

Owner

• Name: Jiri Vanek
• Email: jvanek@rehat.com
  • #fedora-java
  • #fedora-devel
• Release notes owner:

Current status

• Targeted release: Fedora 22
• Last updated: 2014-11-03
• Tracker bug: #1181564
• Original bug: https://bugzilla.redhat.com/show_bug.cgi?id=902086

Detailed Description

The Elasticsearch is fully-featured self-standing opensource indexing server. Many people and many tools do use it. And many people do wont it in Fedora. Aim of this Change is to make elastic search available by simple yum install elasticsearch, and of course enable it as dependence. To build a custom indexing tool on top of elastic search is more easy then current upstream install and download.

Benefit to Fedora

Users of fedora will be able yum install elasticsearch, will be able to use it as dependence and use it in current project without fighting with monolithic upstream installation.

Scope

• Proposal owners:
  • pack Elasticsearch - nearly done - see RHBZ#902086
  • make it somehow works
  • verify it works
  • tune list of crucial depnedences
  • enable Elasticsearch as service - something what have to be decided

• Other developers:
  • This is crucial part of this proposal
  • Elastic search is extremely tuned application, and like it, its dependences must be strictly kept in correct versions
  • Currently known troublemakers:
    • lucene
    • netty3
    • sigar
    • compress-lzf
    • guava (currently needed 18, avaiable 17)

• Release engineering:
  • Nothing I'm aware about.

• Policies and guidelines:
  • Nothing I'm aware about.
  • maybe... fedora is going by way - keep as updated as possible, if it breaks something, that something have to be fixed. This is not best for ES. Maybe ES can et some helping exception?

Upgrade/compatibility impact

original statement - although correct, can be understood to strictly

• [1] when Elasticsearch is updated, all its dependencies must be aligned to exact versions it needs
  • this can involve adding a number of compact packages for ES in case of unsolvable issues
• [2] Non of its dependencies can be updated, unless Elasticsearch is known to work on it
• [3] The list of such restricted despondencies must evolve - see currently known "troublemakers" section

Fixed/more detailed statement:

• [1] It does not mean to to force libraries to update/downgrade to exact version
  • [a] In rawhide, it would be necessary to agree on mayor versions of libraries, that also elasticsearch can be built/run
  • [b] in released version, the minor changes should probably not meter, but would be nice to test ES build/runtime against the,
• [2] Similarly as [1] it does not mean that libraries should be prevented to update.
  • if library is about to be updated, some time should be given to elastic to update. Or better, to let ES upstream to adapt.
  • maintaining compact mayor versions of older packages should be perfectly ok by people around ES

again - I do not wont to lock libraries - I wont to try some kind of cooperation

• [3] If it will appear that this list of libraries is unmaintainable, then I don't know how to proceed (except dropping ES)
  • Still I think this may be tried

How To Test

• yum install elasticsearch
  • start it/use its service
• send json request, accept reply

User Experience

Users are able to use packed Elasticsearch after custom run on some port, or as sevice on known port.

Dependencies

No known project is aiming to be packed with Elasticsource as dependence in close future, but the grow of packages requesting it can be expected. Currently known third party users who will welcome this packed are NSA (yes, that NSA) and Searchisko (Search and content delivery platform behind jboss.org web site).

Contingency Plan 1 - new package

• Contingency mechanism: (What to do? Who will do it?)
  • Although I'm initiator of this, I hope from help from other people around bug 902086 and people around Searchisko.
  • Considering the status of the build, the only expected failure is in dependences - that we will not be able to agree on versions
  • Another failure may raise when some capital dependence change owener and he will not be aware about Elasticsearch boundaries
• If some mayor depndency problems rise
  • create compact packages for them
  • if that fails, drop this feature from f22
• a lot of was agreed here: http://meetbot.fedoraproject.org/fedora-meeting/2015-01-07/fesco.2015-01-07-18.01.log.html I'm not going to rewrite it here, but all in this log should be followed.
• Contingency deadline: Ugh... You tell me...
• Blocks release? no
• Blocks product? no

Contingency Plan 2 - updates

• 1. try to keep package aligned with fedora
  • adapt ES to higher versions, post patches upstream
• 2. if above fails, ask maintainers of freshly updated, suddenly uncompatible, packages to revert and wait a while
• 3. if above fails, create compact package
• If all above fails - drop package from fedora and wait for software collections

Documentation

• https://bugzilla.redhat.com/show_bug.cgi?id=902086 and specfiles linked from here
• upstream pages http://www.elasticsearch.com/products/elasticsearch
• upstream git https://github.com/elasticsearch/elasticsearch/