Ice, Ice Baby
Ice provides a birds-eye view of our large and complex cloud landscape
from a usage and cost perspective. Cloud resources are dynamically
provisioned by dozens of service teams within the organization and any
static snapshot of resource allocation has limited value. The ability
to trend usage patterns on a global scale, yet decompose them down to
a region, availability zone, or service team provides incredible
flexibility. Ice allows us to quantify our AWS footprint and to make
educated decisions regarding reservation purchases and reallocation of
Ice is a Grails project. It consists of three parts: processor, reader
and UI. Processor processes the Amazon detailed billing file into data
readable by reader. Reader reads data generated by processor and
renders them to UI. UI queries reader and renders interactive graphs
and tables in the browser.
More information and screenshots can be found on the project's git page.
What is this repository?
This repository seeks to ease the installation and configuration of Ice. In addition to the application container, this repository configures a nginx proxy which also helps fix URI issues I had when accessing Ice directly. After following these directions you should be able to connect to your server's IP address or FQDN over port 80 and access the Ice application. Additionally, I've supplied an Upstart job script you can leverge to start your containers on boot.
- Sign up for Amazon's programmatic billing access here to receive detailed billing(hourly) reports. Verify you receive monthly billing file in the following format: <accountid>-aws-billing-detailed-line-items-<year>-<month>.csv.zip.
- Docker and Docker Compose installed.
- Create the docker-compose file:
cp docker-compose-template.yml docker-compose.yml
Open docker-compose.yml and configure the Timezone as well as add the AWS Access Key ID and Secret Key that has access to the s3 billing bucket:
ice: build: ice command: | -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses -Duser.timezone=<Your Timezone ie America/New_York> -Dice.s3AccessKeyId=<s3AccessKeyId> -Dice.s3SecretKey=<s3SecretKeyId>
Create the configuration file that will be mounted to the container:
cp ice/assets/sample.properties ice/assets/ice.properties
Open ice.properties and configure a basic setup by updating the following:
# s3 bucket name where the billing files are ice.billing_s3bucketname= # Your company name ice.companyName= # s3 bucket name where Ice can store output files ice.work_s3bucketname= # Your AWS account number. You can also replace "production" with your own identifier ice.account.production=
More information on the configurations can be found on the project's git page.
- When you have completed the previous steps, issue
docker-compose upThis will start the containers in the forground so you can see if there are any errors.
- Once everything looks good and you can access the UI issue
docker-compose up -dto run the containers in the background.
Base Docker Containers
- The nginx container is pulled from the official nginx Docker Hub repository.
- The Ice container's base image is a Java 7 container which is part of an automated build repository that I maintain.
I've included an Upstart job in the
init directory of this repository. This will allow you to start the containers with
start ice and stop them by running
stop ice. This will also start your containers at boot.
init/ice.confto your host's
Edit the the job
vi /etc/init/ice.confand change the path to the docker-compose file
pre-start exec /usr/local/bin/docker-compose -f /path/to/your/docker-compose.yml up -d post-stop exec /usr/local/bin/docker-compose -f /path/to/your/docker-compose.yml stop
Reload the job controller
The version of highstock.js is locked at 4.2.1 due to a breaking change in newer versions.
Thanks for taking the time to build this! I am running into problems bringing the container up locally and on the AWS EC2 Container Service (ECS). The errors are consistent across both. The errors I'm getting seem to stem from a classpath problem:
ice_1 | | Compiling 8 source files
ice_1 | [groovyc] org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
ice_1 | [groovyc] /opt/ice/grails-app/conf/BootStrap.groovy: 47: unable to resolve class com.netflix.ice.reader.ApplicationGroupService
ice_1 | [groovyc] @ line 47, column 1.
ice_1 | [groovyc] import com.netflix.ice.reader.ApplicationGroupService
ice_1 | [groovyc] ^
ice_1 | [groovyc]
ice_1 | [groovyc] /opt/ice/grails-app/conf/BootStrap.groovy: 29: unable to resolve class com.netflix.ice.tag.Account
ice_1 | [groovyc] @ line 29, column 1.
ice_1 | [groovyc] import com.netflix.ice.tag.Account
ice_1 | [groovyc] ^
Have you run across this one before? There haven't been any updates to the Netflix Ice repo and I had it working several weeks ago.
Yes. Turns out that selinux was on and enforcing. Works great now.
I haven't touched this repo in about four months. Were you able to resolve the issue?
I had this running on a test system, and it worked fine. When trying to run this on a dedicated system, I keep getting "ICE_HOME not set" error when running "docker-compose up". Is there a config setting that was added recently?