txn2/rxtx
Queue based data collector and transmitter. Online/offline JSON collection, buffering, queuing.
10K+
txn2/rxtx repository overview
rxtx source
rxtx by txn2
rxtx is a queue based data collector > data transmitter. Useful for online/offline data collection, back pressure buffering or general queuing. rxtx uses bbolt maintained by CoreOs, a single file database for storing messages before they can be sent.
Test on MacOs
Install with brew
brew tap txn2/homebrew-tap
brew install rxtx
Help
rxtx -h
Test Docker Container
Help
docker run --rm -it txn2/rxtx -h
on arm 6/7 based device:
docker run --rm -it txn2/rxtx:arm32v6-1.0.2 -h
Test Source
Help
go run ./rxtx.go -h
Usage of rxtx:
-batch int
Batch size. (default 5000)
-ingest string
Ingest server. (default "http://localhost:8081/in")
-interval int
Seconds between intervals. (default 30)
-maxq int
Max number of message in queue. (default 2000000)
-name string
Service name. (default "rxtx")
-path string
Directory to store database. (default "./")
-port string
Server port. (default "8080")
Start server on 8080
go run ./rxtx.go
Add message to queue
The rxtx services accepts HTTP POST data to an API endpoint in the following form /rx/PRODUCER/KEY/LABEL/.../. One label is required, however as many labels as necessary may be added, separated by a forward slash.
curl -w "\n" -d "{\"generic\": \"$RANDOM\"}" -X POST http://localhost:8080/rx/me/generic_data/generic/test/data
Add message to queue every second
while true; do curl -w "\n" -d "{\"generic\": \"$RANDOM\"}" -X POST http://localhost:8080/rx/me/generic_data/generic/test/data; sleep 1; done
Add 1000 messages to the queue.
time for i in {1..1000}; do curl -w "\n" -d "{\"generic\": \"$RANDOM\"}" -X POST http://localhost:8080/rx/me/generic_data/generic/test/data; done
Building and Releasing
rxtx uses GORELEASER to build binaries and Docker containers.
Test Release Steps
Install GORELEASER with brew (MacOS):
brew install goreleaser/tap/goreleaser
Build without releasing:
goreleaser --skip-publish --rm-dist --skip-validate
Release Steps
- Commit latest changes
- Tag a version
git tag -a v2.0 -m "Version 2.0" - Push tag
git push origin v2.0 - Run:
GITHUB_TOKEN=$GITHUB_TOKEN goreleaser --rm-dist
Resources
License
Tag summary
Content type
Image
Digest
sha256:d0ef11e65…
Size
15.2 MB
Last updated
about 3 years ago
docker pull txn2/rxtx