Cribl puts your IT and Security data at the center of your data management strategy and provides a one-stop shop for analyzing, collecting, processing, and routing it all at any scale. Try the Cribl suite of products and start building your data engine today!
Learn more ›Evolving demands placed on IT and Security teams are driving a new architecture for how observability data is captured, curated, and queried. This new architecture provides flexibility and control while managing the costs of increasing data volumes.
Read white paper ›Cribl Stream is a vendor-agnostic observability pipeline that gives you the flexibility to collect, reduce, enrich, normalize, and route data from any source to any destination within your existing data infrastructure.
Learn more ›Cribl Edge provides an intelligent, highly scalable edge-based data collection system for logs, metrics, and application data.
Learn more ›Cribl Search turns the traditional search process on its head, allowing users to search data in place without having to collect/store first.
Learn more ›Cribl Lake is a turnkey data lake solution that takes just minutes to get up and running — no data expertise needed. Leverage open formats, unified security with rich access controls, and central access to all IT and security data.
Learn more ›The Cribl.Cloud platform gets you up and running fast without the hassle of running infrastructure.
Learn more ›Cribl.Cloud Solution Brief
The fastest and easiest way to realize the value of an observability ecosystem.
Read Solution Brief ›Cribl Copilot gets your deployments up and running in minutes, not weeks or months.
Learn more ›AppScope gives operators the visibility they need into application behavior, metrics and events with no configuration and no agent required.
Learn more ›Explore Cribl’s Solutions by Use Cases:
Explore Cribl’s Solutions by Integrations:
Explore Cribl’s Solutions by Industry:
September 25 | 10am PT / 1pm ET
Hold my beer: lessons from one team’s data pipeline journey
Register ›Try Your Own Cribl Sandbox
Experience a full version of Cribl Stream and Cribl Edge in the cloud.
Launch Now ›Get inspired by how our customers are innovating IT, security and observability. They inspire us daily!
Read Customer Stories ›Sally Beauty Holdings
Sally Beauty Swaps LogStash and Syslog-ng with Cribl.Cloud for a Resilient Security and Observability Pipeline
Read Case Study ›Experience a full version of Cribl Stream and Cribl Edge in the cloud.
Launch Now ›Transform data management with Cribl, the Data Engine for IT and Security
Learn More ›Cribl Corporate Overview
Cribl makes open observability a reality, giving you the freedom and flexibility to make choices instead of compromises.
Get the Guide ›Stay up to date on all things Cribl and observability.
Visit the Newsroom ›Cribl’s leadership team has built and launched category-defining products for some of the most innovative companies in the technology sector, and is supported by the world’s most elite investors.
Meet our Leaders ›Join the Cribl herd! The smartest, funniest, most passionate goats you’ll ever meet.
Learn More ›Whether you’re just getting started or scaling up, the Cribl for Startups program gives you the tools and resources your company needs to be successful at every stage.
Learn More ›Want to learn more about Cribl from our sales experts? Send us your contact information and we’ll be in touch.
Talk to an Expert ›Mobile devices have changed our world. They come with us everywhere and provide invaluable services. One nagging problem is how to get data out of your mobile device. Specifically, logging metrics and events can be a trial. Opening up a public-facing port, managing the log receivers, coding… Wouldn’t it be nice if this was simplified? This article will demonstrate how easy delivering logs can be using Cribl.Cloud and simple HTTP POSTs – for free.
Cribl.Cloud is free for up to 1 terabyte per day. 1000 tasty tasty gigs. Yeaahh, there are no qualifiers here. Use that 1 TB as you see fit. You’ll be able to transform, aggregate, enrich, and route your data in whatever way your requirements dictate. A Leader node, or management interface, will be provided along with a Worker node, which actually receives and processes the data. Key for this exercise: The Worker node is in the Cloud with public facing ports (using TLS of course) ready to receive data from anywhere.
If you haven’t already, get signed up and log in.
On the account landing page, you’ll see a link for “Network Settings” along the top of the page:
Click that, and you’ll be presented with the list of default inputs, ready to be used. One of them is ‘http’. (TLS is enabled, so while it says http, it will actually use https. Score.) Note the link provided. I’d recommend copying it for later referral:
Now click that ‘http’ link on the left, and it will take you into the Stream management interface, directly to the source definition. The default HTTP source includes a token that allows access to the endpoint. If you click the eye icon on the right side of the field, it will toggle hiding/showing the value. Show it, then copy it. We’ll need that (along with the address from above).
While in the Source config screen, click the “Connected Destinations” link on the left, and change it to QuickConnect. Save.
The default configuration will be sending any data received to our old friend Davie Null. This is fine for testing, but you’ll eventually want to set-up one or more actual destinations. Cribl has a diverse selection available, including object stores, SIEMs, and open protocols. For a simple verification during this exercise, I used Sumo Logic. They have a free trial, sign-up was easy, and as with most configs in Cribl, the destination set-up is dead simple:
That said, a detailed destination discussion is outside the scope of this article. The Cribl’s docs are fantastic though.
Under Routing, go to QuickConnect (see the docs).
You should see your http
input there on the left, and your destinations on the right. Connect them with a simple drag-n-drop. When the Pipeline option pops up, select Passthru.
Click Save, then the Commit button in the upper right. Give your commit a message like “set-up route and destination for HTTP-based log delivery”. Click Commit, then Click the Deploy button.
Cribl.Cloud provided an endpoint (source) to send data to. We created a route that connects that source to a named destination. We could define that destination to send to an analysis service, but for this demo we’re dumping the data to devnull. We could also get trickier with the Data Routes instead of QuickConnect, and/or a Pipeline to enrich, filter, and transform the data, but we’ll leave those for a future how to blog.
Now that the plumbing is in place, let’s get some data flowing.
Using the token and the URL you grabbed in the first steps above, we can construct a simple curl command to send data.
Before we run it, start a Live Data capture in the Cribl interface. Go back to the Data -> Sources -> HTTP source, open up the config, and then click on the Live Data tab.
⚠️This is one of the handiest features of Cribl. We can see what is being delivered to us right off the wire! We can also capture and save this sample so as we build out rulesets we have real data to reference and verify impact. Getting Data In has never been easy. Cool!
Now pop into a terminal window with curl available. Most Linux distros have it by default. macOS also ships with it by default. Windows will need to have it installed. Run this command:
curl -H "Authorization: Bearer <token>" \
-X POST -d '{"event": "test", "field1":"value1"}' \
<url>/services/collector
Substitute the values of the token and the URL from above into the correct slots. A successful post will result in: {“text”:”Success”,”code”:0}
Immediately, you should see data show on the Live Data capture you have running in the browser. Congratulations, you just created your first logging event in Cribl.Cloud. You’ll notice the contents of event are placed in _raw. Any other fields are added to the event metadata. Think of an event as an envelope. Inside the envelope is _raw, the message you’re sending. Written on the outside of the envelope is metadata. The message might be something like “failed to login, bad password”. Maybe the account id and version are in the metadata. A slightly more practical example might look something like this:
curl -H "Authorization: Bearer <token>" \
-X POST -d '{"event": "failed to login, bad password",\
"userid":"thedude", "version":"1.42"}' \
<url>/services/collector
Resulting in an event like so:
We set-up the plumbing ✅
We tested ingest with sample data ✅
To bake this into your app, IOT device, or utility, you’ve got the key required bits:
For a truly robust and production-ready system you’d want to embellish this set-up with some sort of failsafe in case the logging endpoint isn’t available or the network is down.
Below I’ve included some real samples of the basic set-up using Apple iOS devices.
This is a pretty cool demo put together by Cribl Community user @starcher. It automatically logs data (location, battery level, etc) to Cribl.Cloud when you fire off a Shortcut. Originally, he had used it to send to Splunk HEC directly, but converting it to work with Cribl took minimal effort, and Cribl allows so much more flexibility with his data.
The set-up is shown below in screencaps because Shortcuts is very visual/GUI driven. There’s not much text-based code.
First, he collects up the bits of info he wants to send and builds the dictionary which will be the payload. Within this main dictionary, he also created an event dictionary (next pic) which will be the event (ie, envelope content). At the end of this screencap, he runs Cribl Receiver.
The event dict:
And the definition for Cribl Receiver:
With these few lines of code, you could enable logging directly from deployed mobile apps. Refer to Apple’s Swift docs for more detailed info.
Here we create the chunk of data to post. We create the struct spec, a new object based on it, and then we convert that to JSON, as required by the endpoint.
struct MyLogEvent: Codable {
let userid: String
let version: String
let msg: String
}
let eventdata = MyLogEvent(
userid: "thedude",
version:"1.42",
msg: "failed to login, bad password")
guard let uploadData = try? JSONEncoder().encode(order) else {
return
}
Now we are preparing the request. You’ll need the URL and token for this block of code.
let url = URL(string: "<enter your url here>/services/collector")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
request.setValue("<enter your token here>", forHTTPHeaderField: "Authorization")
Finally, using the previous code, we make a POST request.
let task = URLSession.shared.uploadTask(with: request, from: uploadData) { data, response, error in
if let error = error {
print ("error: \(error)")
return
}
guard let response = response as? HTTPURLResponse,
(200...299).contains(response.statusCode) else {
print ("server error")
return
}
if let mimeType = response.mimeType,
mimeType == "application/json",
let data = data,
let dataString = String(data: data, encoding: .utf8) {
print ("got data: \(dataString)")
}
}
task.resume()
Using Cribl unshackles you from your analysis tool. With a few clicks, you can change how your data is routed, filtered, transformed and archived. Using Azure object storage (blob)? Sure, send it there. Using Event Hub? New Relic? QRadar? Splunk? Absolutely. Want to use multiple destinations, sending a copy to each place? YES! Easy to do.
With Cribl the fasten seatbelt sign is off! Your data is free to move about the cabin.
Sign-up for your free Cribl.Cloud Suite now to get 1 TB/day of ingest for free.
The fastest way to get started with Cribl Stream and Cribl Edge is to try the Free Cloud Sandboxes.
Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.
Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari
Got one of those handy?