title | tags | translate | metaDescription | redirects | freshnessValidatedDate | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Fluent Bit plugin for log forwarding |
|
|
Install and configure the New Relic logging plugin for Fluent Bit, so you can use enhanced log management capabilities. |
|
never |
If your log data is already being monitored by Fluent Bit, you can use our Fluent Bit output plugin to forward and enrich your log data in New Relic.
Forwarding your Fluent Bit logs to New Relic will give you enhanced capabilities to collect, process, explore, query, and alert on your log data.
We have published a container with the plugin installed. It serves as a base image to be used by our Kubernetes integration. We recommend you use this base image and layer your own custom configuration files.
To forward your logs from Fluent Bit to New Relic:
- Make sure you have:
- Install the Fluent Bit plugin.
- Configure the Fluent Bit plugin.
- Test the Fluent Bit plugin.
- Generate some traffic and wait a few minutes, then check your account for data.
To install the Fluent Bit plugin:
-
Navigate to New Relic's Fluent Bit plugin repository on GitHub. 2. From the repository page, clone or download the repository. 3. Run the following command to build your plugin:
cd newrelic-fluent-bit-output && make all
-
Store
out_newrelic.so
orout_newrelic_winXX.dll
at a location that can be accessed by thefluent-bit
daemon.
Before you upgrade your Fluent Bit plugin, run the following NRQL query to find the current versions of the output plugin being used in your system:
FROM K8sContainerSample
SELECT latest(containerImage)
WHERE podName like '%newrelic-logging%'
FACET clusterName
To upgrade, follow the installation instructions or grab the latest pre-compiled version from our GitHub repository.
New Relic has a Fluent Bit output plugin to forward your logs to New Relic log management. This plugin is also provided in a standalone Docker image that can be installed in a Kubernetes cluster in the form of a DaemonSet, also known as the Kubernetes plugin.
You can install it in your cluster using our Helm chart in two ways.
Although the newrelic-logging chart works as a standalone, we recommend installing it as part of the nri-bundle chart.
The best way to install this is through our guided installation process. This guided install can generate the Helm 3 commands required to install it (see "Helm 3").
Alternately, you can install it manually using Helm, by running this command to install the repo:
helm repo add newrelic https://helm-charts.newrelic.com
To update the repo you can run:
helm repo update newrelic
Go here for uninstallation instructions.
Fluent Bit needs to know the location of the New Relic plugin and the New Relic to output data to New Relic. To configure your Fluent Bit plugin:
Pay attention to white space when editing your config files. Be sure to use four spaces to indent and one space between keys and values.-
Locate or create a
plugins.conf
file in your plugins directory. -
In the
plugins.conf
file, add a reference toout_newrelic.so
, adjacent to yourfluent-bit.conf
file:[PLUGINS] Path /PATH/TO/newrelic-fluent-bit-output/out_newrelic.so
-
In the
fluent-bit.conf
file, add the following line under theservice
block:[SERVICE] # This is the main configuration block for fluent bit. # Ensure the follow line exists somewhere in the SERVICE block Plugins_File plugins.conf
-
At the bottom of the
fluent-bit.conf
file, add the following to set up the input, filter, and output sections. Replace the placeholder text with your :[INPUT] Name tail Tag my.tag Path /PATH/TO/YOUR/LOG/FILE # If you have multiple sources, just add another [INPUT] section like this: [INPUT] Name tail Tag my.other.tag Path /PATH/TO/SOME/OTHER/LOG/FILE # Having multiple [FILTER] blocks allows you to control the flow of changes as they read top down. [FILTER] Name modify # Here we only match on one tag, my.tag, defined in the [INPUT] section earlier Match my.tag # Below, we're renaming the host.cpu attribute to CPU Rename host.cpu CPU [FILTER] Name record_modifier # Match on all tags, *, so all logs get decorated per the Record clauses below. Record adds attributes + their values to each record. Match * # Adding a logtype attribute ensures your logs will be automatically parsed by our built-in parsing rules Record logtype nginx # Add the server's hostname to all logs generated Record hostname ${HOSTNAME} [OUTPUT] Name newrelic Match * licenseKey YOUR_LICENSE_KEY
-
Restart your Fluent Bit instance with the following command:
fluent-bit -c /PATH/TO/fluent-bit.conf
To test if your Fluent Bit plugin is receiving input from a log file:
-
Run the following command to append a test log message to your log file:
echo "test message" >> /PATH/TO/YOUR/LOG/FILE
-
Search our logs UI for
test message
.
For more options, see the Fluent Bit modify filter documentation and our documentation to forward your logs using the infrastructure agent.
Once you have installed and configured the Fluent Bit plugin, you can use the following attributes to configure how the plugin sends data to New Relic:
<th>
Description
</th>
</tr>
<td>
The <InlinePopover type="licenseKey"/>. Use either `licenseKey` (<DNT>**recommended**</DNT>) or `apiKey`, not both. Default: `none`
</td>
</tr>
<tr>
<td>
`maxBufferSize`
</td>
<td>
The maximum size the payloads sent, in bytes. Default: `256000`
</td>
</tr>
<tr>
<td>
`maxRecords`
</td>
<td>
The maximum number of records to send at a time. Default: `1024`
</td>
</tr>
<tr>
<td>
`apiKey`
</td>
<td>
<DNT>**Deprecated.**</DNT> Takes a New Relic [Insights insert key](/docs/insights/insights-data-sources/custom-data/send-custom-events-event-api#register), but using the `licenseKey` attribute is preferred. Use either `licenseKey` or `apiKey`, not both.
</td>
</tr>
<tr>
<td>
`endpoint`
</td>
<td>
Defaults to `https://log-api.newrelic.com/log/v1` - If using an EU key needs to be set to `https://log-api.eu.newrelic.com/log/v1`
</td>
</tr>
Key |
---|
`licenseKey` |
If everything is configured correctly and your data is being collected, you should see log data in both of these places:
- Our logs UI
- Our tools for running NRQL queries. For example, you can execute a query like this:
SELECT * FROM Log
If no data appears after you enable our log management capabilities, follow our standard log troubleshooting procedures.
Explore logging data across your platform with our logs UI.
- Get deeper visibility into both your application and your platform performance data by forwarding your logs with our logs in context capabilities.
- Set up alerts.
- Query your data and create dashboards.
To disable log forwarding capabilities, follow standard procedures in Fluent Bit documentation. You do not need to do anything else in New Relic.