From Fedora Project Wiki
mNo edit summary
 
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''''Javi Roman:''''' [https://twitter.com/javiromanrh Twitter] [http://es.linkedin.com/in/javiroman Linkedin] [http://www.viewbug.com/member/javiroman Photography]
'''''Javi Roman:''''' [https://twitter.com/javiromanrh Twitter] [http://es.linkedin.com/in/javiroman Linkedin] [http://www.viewbug.com/member/javiroman Photography]


= Fedora Big Data Package Ecosystem =
This is the personal page of [https://fedoraproject.org/wiki/User:Jromanes Javi Roman] current Apache Mesos Fedora Package maintainer. In this page the Apache Mesos Package is tracked in order to maintain the permanence in Fedora repositories.


{| class="wikitable" style="color:black; background-color:#CCFFFF;" cellpadding="10"
= Apache Mesos Package Sources =
|+ Fedora Hosted Packages
! Package !! Packaged Version !! Upstream Version !! Sources  
|-
|'''Apache Hadoop'''
|2.4.1
|2.7.0
|http://pkgs.fedoraproject.org/cgit/hadoop.git/
|-
|'''Apache HBase'''
|0.98.3
|1.0.1
| http://pkgs.fedoraproject.org/cgit/hbase.git/
|-
|'''Apache Hive'''
|0.12.2
|1.1.0
|http://pkgs.fedoraproject.org/cgit/hive.git/
|-
|'''Apache Pig'''
|0.13.10
|0.14.0
|http://pkgs.fedoraproject.org/cgit/pig.git/
|-
|'''Apache Zookeeper'''
|3.4.6
|3.4.6
|http://pkgs.fedoraproject.org/cgit/zookeeper.git/
|-
|'''Apache Oozie'''
|4.0.1
|4.1.0
|http://pkgs.fedoraproject.org/cgit/oozie.git/
|-
|'''Apache Ambari'''
|1.5.1
|2.0.0
|http://pkgs.fedoraproject.org/cgit/ambari.git/
|-
|'''Apache Accumulo'''
|1.6.1
|1.6.2
|http://pkgs.fedoraproject.org/cgit/accumulo.git/
|-
|'''Apache Mesos'''
|0.22.1
|0.22.1
|http://pkgs.fedoraproject.org/cgit/mesos.git/
|-
|'''Apache Solr'''
|4.10.4
|5.1.0
|http://pkgs.fedoraproject.org/cgit/solr.git/
|-
|'''Apache Spark'''
|0.9.1
|1.3.1
|http://pkgs.fedoraproject.org/cgit/spark.git/
|-
|'''AMPLab Tachyon'''
|0.99
|0.6.4
|http://pkgs.fedoraproject.org/cgit/tachyon.git
|-
|
|
|
|
|}


{| class="wikitable" style="color:black; background-color:#CCFFFF;" cellpadding="10"
The official package source code is maintained in the Fedora Package Sources site:
|+ On going packages
! Package !! Packaged Version !! Upstream Version !! Status !! Sources  
|-
|'''Apache Flume'''
|1.5.0
|1.5.0
|Partially supported
|https://github.com/fedora-bigdata-rpms/flume-rpm
|-
|'''Cloudera Kite SDK'''
|1.0.0
|1.0.0
|
|https://gil.fedorapeople.org/kite.spec
|-
|'''Apache Crunch'''
|0.11.0
|0.11.0
|
|https://github.com/fedora-bigdata-rpms/crunch-rpm
|-
|'''Apache Tez'''
|0.5.3
|0.6.0
|
|https://github.com/fedora-bigdata-rpms/tez-rpm
|-
|'''Apache Kafka'''
|0.8.0
|0.8.2.1
|
|https://github.com/fedora-bigdata-rpms/kafka-rpm
|-
|'''Apache Tajo'''
|0.10.0
|0.10.0
|
|https://gil.fedorapeople.org/tajo.spec
|-
|'''Apache Jena'''
|2.13.0
|2.13.0
|
|https://gil.fedorapeople.org/jena.spec
|-
|'''Cascading'''
|2.6.3
|2.6.3
|
|https://gil.fedorapeople.org/cascading.spec
|-
|
|
|
|
|
|}


= Apache Flume package status =
[https://src.fedoraproject.org/rpms/mesos Fedora Package Sources]


===Package status===
= Package History =


The package builds with this assumptions (we are working on this issues)
Review Request:
[https://bugzilla.redhat.com/show_bug.cgi?id=1749383 Fedora 30 Package Review]


* The code is not ready for Thrift v0.9.1 available in Fedora 21, however Flume code can builds using legacy Thrift built-in code available in the upstream Flume TGZ.
{| class="wikitable" style="color:black; background-color:#CCFFFF;" cellpadding="10"
* Disable ElasticSearch Sink
! Package !! Description !! Packaged <br> Version !! Upstream <br> Version !! Sources !! Who !! Notes
* Disable Morphline Solr Sink
* Disable Twitter Source
* Disable Kite Dataset Sink
 
===Testing the package===
 
git clone https://github.com/fedora-bigdata-rpms/flume-rpm.git
cd flume-rpm
spectool -g flume.spec
rpmbuild -bs --nodeps --define "_sourcedir ." --define "_srcrpmdir ." flume.spec
sudo mock flume-1.5.2-1.fc21.src.rpm
 
===Dependency packages===
 
* In order to build Flume with full features those are the dependency packages and their status:
 
{| class="wikitable" style="color:green; background-color:#ffffcc;" cellpadding="10"
! Package !! Bugzilla !! Status
|-
|-
|irclib
| '''Apache Mesos'''
|<s>{{bz|976049}}</s>
| Cluster manager for sharing distributed application frameworks
|Package is available in Rawhide and in Fedora 21 as an update
| 1.8.1
|-
| 1.9.0
|mapdb
| [https://src.fedoraproject.org/rpms/mesos mesos.git]
|<s>{{bz|1178861}}</s>
|
|Package is available in Rawhide and in Fedora 21 as an update
| [[Changes/ApacheMesos | Mesos]]
|-
|asynchbase
|No BZ ticket
|No added for revision in Bugzilla
|-
|suasync
|No BZ ticket
|aynchbase dependency. No added for revision in Bugzilla
|-
|kite
|{{bz|1179355}}
|Patched in order to support Fedora Guava version (partial support).
|-
|parquet
|<s>{{bz|1073017}}</s>
|kite package dependency. Package is available in Rawhide and was submitted to Fedora 22 and 21 as an update
|-
|parquet-format
|<s>{{bz|1073014}}</s>
|parquet package dependency. Package is available in Rawhide and was submitted to Fedora 22 and 21 as an update
|-
|maxmind-db-java
|<s>{{bz|1179309}}</s>
|kite package dependency. Package is available in Rawhide and in Fedora 21 as an update
|-
|ua-parser-java
|<s>{{bz|1179342}}</s>
|kite package dependency. Package is available in Rawhide and in Fedora 21 as an update
|-
|elasticsearch
|<s>{{bz|902086}} {{bz|1181564}}</s>
|Package is available in Rawhide and in Fedora 22 as an update
|}
|}
= Apache Storm package status =
[https://github.com/fedora-bigdata-rpms/storm-rpm sources]
= Apache Kafka package status =
[https://github.com/fedora-bigdata-rpms/kafka-rpm sources]
Apache Kafka is a distributed publish-subscribe messaging system persistent oriented with O(1) disk structures that provide constant time performance even with many TB of stored messages.
Apache Kafka is based on '''Scala''' language. Scala uses '''sbt (Simple Build Tool)''' for builds, it's the de facto build tool for the Scala community. Sbt is similar to Apache Ant, and uses '''Apache Ivy''' (a sub-project of the Apache Ant project) for resolving project dependencies.
We have two methods for scala based project RPM building:
* Building packages with sbt and the '''climbing-nemesis script''' (a tool to make a temporary Ivy repository from installed Fedora packages)
  [[SIGs/bigdata/packaging/Sbt]]
  [http://chapeau.freevariable.com/2014/02/sbt-is-in-fedora-20.html sbt is in Fedora 20 ]
  [http://pkgs.fedoraproject.org/cgit/sbinary.git/tree/sbinary.spec Example of climbing-nemsis usage]
* Building packages with sbt and '''xmvn’s Ivy resolution support'''
  [http://chapeau.freevariable.com/2013/08/making-fedora-a-better-place-for-scala.html Making Fedora a better place for Scala]
  [https://bugzilla.redhat.com/show_bug.cgi?id=1012612 improved Fedora support for Ivy]
  [[SIGs/bigdata/packaging/Scala]]
  [[Changes/ImprovedScalaEcosystem]]
  [[Changes/ImprovedIvyPackaging]]
Scala project based on sbt are broken in Fedora23-rawhide, the pending bugs here:
[https://lists.fedoraproject.org/pipermail/java-devel/2014-December/005430.html Removing depmap support in Fedora 23]
[https://bugzilla.redhat.com/show_bug.cgi?id=1107277 sbt: FTBFS in rawhide]
[https://bugzilla.redhat.com/show_bug.cgi?id=1155699 sbt: broken hawtjni-runtime-1.8.jar symlink]
= Apache Tez package status =
[https://github.com/fedora-bigdata-rpms/tez-rpm sources]
= Apache Crunch package status =
[https://github.com/fedora-bigdata-rpms/crunch-rpm sources]

Latest revision as of 09:27, 26 January 2020

Javi Roman: Twitter Linkedin Photography

This is the personal page of Javi Roman current Apache Mesos Fedora Package maintainer. In this page the Apache Mesos Package is tracked in order to maintain the permanence in Fedora repositories.

Apache Mesos Package Sources

The official package source code is maintained in the Fedora Package Sources site:

Fedora Package Sources

Package History

Review Request: Fedora 30 Package Review

Package Description Packaged
Version		 Upstream
Version		 Sources Who Notes
Apache Mesos Cluster manager for sharing distributed application frameworks 1.8.1 1.9.0 mesos.git Mesos
Retrieved from "https://fedoraproject.org/w/index.php?title=User:Jromanes&oldid=564664"