C3PO version 0.6

.dockerignore

@@ -0,0 +1,3 @@
+Dockerfile
+.idea/
+*/target/*

.gitignore

@@ -3,6 +3,7 @@
 .settings/
 .DS_Store
 target/
+*/target/*
 data/
 bin/
 c3po-core/output.xml
@@ -12,3 +13,8 @@ c3po-webapi/war/
 */*.csv
 */logs/
 */log
+*.iml
+.idea/
+# vagrant related local settings, should never end up in the repo
+.vagrant_settings/maven_settings.xml
+.vagrant_settings/proxy_settings.conf

.travis.yml

@@ -1,14 +1,18 @@
-language: java
-
-branches:
-  only:
-    - master
-    - integration
-
+language: scala
+scala:
+- 2.10
 jdk:
-  - oraclejdk6
-  - oraclejdk7
-  - openjdk6
-  - openjdk7
+- oraclejdk8
+cache:
+  directories:
+  - "$HOME/.ivy2/cache"
+before_cache:
+- rm -rf $HOME/.ivy2/cache/com.typesafe.play/*
+- rm -rf $HOME/.ivy2/cache/scala_*/sbt_*/com.typesafe.play/*
+- find $HOME/.ivy2/cache -name "ivydata-*.properties" -print0 | xargs -n10 -0 rm
 
-services: mongodb
+# See https://blog.travis-ci.com/2014-03-13-slack-notifications/
+# created with travis encrypt command line tool
+notifications:
+  slack:
+    secure: 3BJCD1KtQdPw+Q/eg4wN4DYHHXvS8/YIdNA1xdwPREuCe6rn4khhTu3HcREI07rG3wfQhphy+f1bs+A3K2h9SGVoa4tslng7Bg2jFlf50pXFJZhhXcHQxCApxhj93TP54SQxRFtYLSOueJa6YRWnqxJMZpuMnGOU9cY3iuYbIaExTncjAZdkLsZbFaJtGhI2PqgIyEPEGo8CMZ1EQs2EP+vWKAS0rsKYQNHPd2hp7Z1cHzU0w8SNOFmkgy11J/NFe/Of3Bt67PfIMUnxA61hB/Xl5llqCkWmf5shntyAGCo8bxqWHlK+O8+ZU49EODl+kChJklcQ7btPB7vc3AXFpDllegDg4d8dWszwbo7yX3zEjr4iQtv2j5QW6euHMW9LBkBiLBuAg5vEB2ERQDsz888YL2djRtATuvwS77HSznmnmsENYTsOdm+mjm+x2R9k1uZ68+z3qBoLhsVpsxRakJBuwcJO8EE0NUTQOTm/ftgJ0NQpf7TWfByOqDCv/bZeZU/lK71NLKARJbe5bC2QW0oD5LjINITT+nd5UFmkirmy10EVFiNvF+GLKNfSk3pVQ2lUVKrF22qI9d+wR+Oy5aZBNxKK7tySKiT2XNbgNdNzmX66bV2ib+zm4A3Te5dvetNMIFy8kcU34LTe5NIbZt/efGagpo+1ijhLS6gBiKk=

Dockerfile

@@ -0,0 +1,65 @@
+# syntax = docker/dockerfile:1.2
+
+FROM ubuntu:18.04
+
+RUN rm -f /etc/apt/apt.conf.d/docker-clean
+
+
+
+
+RUN apt-get update \
+    && apt-get -y install apt-transport-https \
+    && apt-get update \
+    && apt-get install -yqq --no-install-recommends openjdk-8-jdk screen \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY . /c3po
+WORKDIR /c3po
+
+RUN ["/bin/bash", "-c", "./sbt clean compile assembly dist"]
+
+
+
+
+
+FROM ubuntu:18.04
+
+
+RUN apt-get update \
+    && apt-get -y install apt-transport-https gnupg curl \
+    && curl -fsSL https://www.mongodb.org/static/pgp/server-3.6.asc | apt-key add - \
+    && echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/3.6 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.6.list \
+    && apt-get update \
+    && apt-get install -yqq --no-install-recommends mongodb-org openjdk-8-jdk screen unzip
+
+RUN apt-get clean autoclean \
+    && apt-get autoremove --yes \
+    && rm -rf /var/lib/apt \
+    && rm -rf /var/lib/dpkg \
+    && rm -rf /var/cache/apt
+
+RUN rm -f /etc/apt/apt.conf.d/docker-clean
+RUN mkdir /c3po
+WORKDIR /c3po
+
+COPY --from=0 /c3po/c3po-cmd/target/scala-2.11/c3po-cmd-assembly-0.1-SNAPSHOT.jar ./c3po-cmd.jar
+COPY --from=0 /c3po/c3po-webapi/target/universal/c3po-webapi-0.1-SNAPSHOT.zip ./c3po-webapi.zip
+RUN unzip c3po-webapi.zip && rm -rf c3po-webapi.zip
+
+EXPOSE 9000
+
+RUN echo "#!/bin/bash \n\
+set -e \n\
+echo The number of files found in /data/FITS: \n\
+find /data/FITS -type f | wc -l \n\
+echo 'Now, C3PO will import metadata from FITS files..' \n\
+mkdir -p /data/db \n\
+nohup mongod --dbpath /data/db & \n\
+sleep 10 \n\
+java -jar /c3po/c3po-cmd.jar gather -c indocker -i /data/FITS -r \n\
+ls \n\
+pwd \n\
+./c3po-webapi-0.1-SNAPSHOT/bin/c3po-webapi -Dplay.crypto.secret=abcdefghijk \n\
+" >> /import.sh
+
+ENTRYPOINT ["bash","/import.sh"]

README.md

@@ -1,51 +1,30 @@
-C3PO
-===================================================
-[![Build Status](https://travis-ci.org/peshkira/c3po.png?branch=master)](https://travis-ci.org/peshkira/c3po)
-
-Clever, Crafty, Content Profiling of Objects (c3po) is a software tool, which uses meta data extracted from
-files of a digital collection as input to generate a profile of the content set. It is designed in a way so
-that different meta data formats originating from different tools can be easily integrated. Currently it
-supports FITS meta data and Apache TIKA meta data.
-
-The tool follows a three part profiling process and provides facilities for data export and further
-analysis of the content, such as helpful visualisations of the meta data characteristics, partitioning
-of the collection into homogeneous sets based on any known characteristic. For each chosen partition of
-the content, a special machine-readable profile can be generated that contains aggregations and
-distributions for many of the properties. The profile optionally contains the set of chosen sample objects
-that are representative.
-
-Releases
-------------------------
-Please refer to [BinTray](http://dl.bintray.com/peshkira/c3po)
-
-Setup
-------------------------
-Please refer to the [Usage Guide](https://github.com/peshkira/c3po/wiki/Usage-Guide).
-
-Development
-------------------------
-Please refer to the [Dev Guide](https://github.com/peshkira/c3po/wiki/Development-Guide).
-
-Screenshot
-------------------------
-![Collection Overview](https://dl.dropbox.com/u/8290338/blog/c3po_overview.png "Collection Overview")
-
-More Information
-------------------------
-You can find more information in the following links:
-- [Website](http://ifs.tuwien.ac.at/imp/c3po)
-- [Blog Post](http://www.openplanetsfoundation.org/blogs/2012-11-19-c3po-content-profiling-tool-preservation-analysis)
-- [Screencast](https://vimeo.com/53069664)
-
-Road Map
-------------------------
-* consolidate based on resource name
-* bundle optional! FITS execution in c3po (to make it easier for demo purposes)
-* create a consistent REST API
-* refactor the web app to use the new REST API and the new core
-* read data from memory instead of file system and allow adaptors to skip the 
-memory read 
-* make use of a controlled vocabulary for properties. If nothing better exists, then use FITS as default.
-* implement HBASE backend
-* ...
-* scale to half a billion objects
+# C3PO: Clever, Crafty Content Profiling of Objects
+
+Analyze your content with C3PO.
+
+### How to install and use
+
+Installation and usage instructions are available in our wiki [here](https://github.com/datascience/c3po/wiki).
+
+### Troubleshooting
+
+If you encounter any problems, please let us know by submitting an issue [here](https://github.com/datascience/c3po/issues?state=open).
+
+### Licence
+
+C3PO is released under [Apache version 2.0 license](LICENSE.txt).
+
+### Acknowledgements
+
+Part of this work was supported by the Vienna Science and Technology Fund (WWTF) through project ICT12-046 (BenchmarkDP) and by the 7th Framework Program, IST, through the SCAPE project, Contract 270137.
+
+### Support
+
+This tool is supported by the [Open Planets Foundation](http://www.openplanetsfoundation.org). 
+
+### Roadmap
+
+* Conflict resolution
+* A controlled vocabulary for properties
+* Templating mechanism 
+