Axibase Collector
Overview
Axibase Collector is a Java-based ETL application that queries external data sources on a defined schedule and uploads the data as series, properties, messages, and files into the Axibase Time Series Database (ATSD).
The Collector supports data markers to minimize the load on the source systems as well as list-based automation to efficiently copy data from multiple sources with re-usable configurations.
Use Cases
- Download a CSV/TSV file from a URL or an FTP server and upload it into ATSD.
- Offload data from a relational database for long-term storage in ATSD.
- Query a web service, convert its JSON output into tabular format, and upload it into ATSD.
- Historize metrics by querying current values and retaining their history in ATSD.
- Collect key performance metrics from Java applications (JMX), network devices (SNMP), etc.
Supported Databases
- Oracle
- IBM DB2
- Microsoft SQL Server
- MySQL
- PostgreSQL
- Sybase
- Apache Derby
- Vertica
- ATSD
- SAP HANA
- OSISoft PI Data Archive
- OSISoft PIOleDBEnt
- Custom
Supported Data/Network Protocols
- JDBC
- SNMP
- JMX
- ICMP
- TCP
- HTTP
Supported File Formats
- CSV
- TSV
- Text
- JSON
Supported Schemas
Supported APIs
- Kafka
- Amazon Web Services CloudWatch
- Docker Engine
- HP OVPM (Performance Manager)
Installation
- Requirements
- Installation:
- Installation on host
- Installation in Docker container
- Installation on Kubernetes
- Administrator Account
- Collector account in ATSD
- ATSD Server Connection
Jobs
Automation
Job Types
| Type | Description |
|---|---|
| AWS | Collects AWS statistics using CloudWatch API. |
| Docker | Collects container, image, and volume statistics using Docker Engine API. |
| File | Downloads CSV/TSV files from remote servers or local file system and uploads them into ATSD for parsing. Supports HTTP/s, FTP, SFTP, SCP, and FILE protocols. |
| HTTP | Executes HTTP requests or Web Driver scripts and stores response status and response times as metrics. |
| ICMP | Pings hostnames/IP addresses and stores response status. |
| JDBC | Executes SQL queries against relational databases. Converts rows into series, property, or message commands. |
| JMX | Collects MBean attribute values from Java applications. |
| JSON | Downloads JSON files, parses the documents and converts JSON fields into series, property, and message commands using JSONPath. |
| Kafka | Reads JSON messages from Kafka brokers, parses the messages and converts them into series, property, and message commands. |
| OVPM | Offloads statistics from HP OpenView Performance Manager. |
| PI | Extracts PI points archive data via JDBC driver. |
| SNMP | Queries SNMP devices using built-in and custom MIB files. |
| Socrata | Downloads JSON documents published in Socrata schema and converts them into series, property, and message commands. |
| TCP | Connects to hostnames/IP addresses and stores connection status. |
Administration
Examples
| Name | Job Type | Description |
|---|---|---|
| ActiveMQ | JMX | Collect metrics about brokers, queues, pub/sub topics |
| Derby Database | JMX | Collect uptime metrics for the database |
| File | File | Download CSV files |
| HP OpenView | OVPM | Offload CODA metrics from OVPM |
| Jetty | JMX | Collect sessions, requests, status from the Jetty server |
| JSON | JSON | Download JSON file and convert it to CSV format |
| JVM | JMX | Collect key JVM performance metrics |
| MySQL | JDBC | Collect database performance metrics |
| nginx | File | Collect key web server metrics for nginx |
| nginx-plus | File | Collect extended web server metrics for nginx-plus |
| Oracle EM | JDBC | Offload incremental database and application metrics collected by Oracle EM |
| PI | PI | Copy incremental tag values from PI Data Archive |
| PostgreSQL | JDBC | Collect database performance metrics |
| SCOM | JDBC | Offload incremental server metrics collected by SCOM |
| Socrata | Socrata | Download and parse datasets published in Socrata format |
| SolarWinds | JDBC | Offload incremental network and server metrics collected by SolarWinds |
| Tomcat | JMX | Collect key container metrics exposed by Tomcat |
| VMware | JDBC | Offload incremental cluster, host, VM metrics collected by VMware vCenter |