This guide explains how to configure Apache Log4j to send logs to Axiom
Log4j is a Java logging framework developed by the Apache Software Foundation and widely used in the Java community. This page covers how to get started with Log4j, configure it to forward log messages to Fluentd, and send logs to Axiom.
Log4j is a flexible and powerful logging framework for Java applications. To use Log4j in your project, add the necessary dependencies to your pom.xml file. The dependencies required for Log4j include log4j-core, log4j-api, and log4j-slf4j2-impl for logging capability, and jackson-databind for JSON support.
A Socket appender that sends logs to Fluentd, running on localhost:24224. Is uses JSON format for the log messages, which makes it easier to parse and analyze the logs later in Axiom.
A Console appender that prints logs to the standard output,
Log4j supports various log levels, allowing you to control the verbosity of your logs. The main log levels, in order of increasing severity, are the following:
TRACE: Fine-grained information for debugging.
DEBUG: General debugging information.
INFO: Informational messages.
WARN: Indications of potential problems.
ERROR: Error events that might still allow the app to continue running.
FATAL: Severe error events that might lead the app to cancel.
In the configuration above, the root logger level is set to INFO which means it logs messages at INFO level and above (WARN, ERROR, and FATAL).
To set the log level, create a simple Java class to demonstrate these log levels. Create a new file named App.java in the src/main/java/com/example directory with the following content:
java
package com.example;import org.apache.logging.log4j.LogManager;import org.apache.logging.log4j.Logger;import org.apache.logging.log4j.ThreadContext;import org.apache.logging.log4j.core.config.Configurator;import org.apache.logging.log4j.Level;import java.util.Random;public class App { // Define loggers for different purposes private static final Logger logger = LogManager.getLogger(App.class); private static final Logger securityLogger = LogManager.getLogger("SecurityLogger"); private static final Logger performanceLogger = LogManager.getLogger("PerformanceLogger"); public static void main(String[] args) { // Configure logging levels programmatically configureLogging(); Random random = new Random(); // Infinite loop to continuously generate log events while (true) { try { // Simulate various logging scenarios simulateUserActivity(random); simulateDatabaseOperations(random); simulateSecurityEvents(random); simulatePerformanceMetrics(random); // Simulate a critical error with 10% probability if (random.nextInt(10) == 0) { throw new RuntimeException("Simulated critical error"); } Thread.sleep(1000); // Sleep for 1 second } catch (InterruptedException e) { logger.warn("Sleep interrupted", e); } catch (Exception e) { logger.error("Critical error occurred", e); } finally { // Clear thread context after each iteration ThreadContext.clearAll(); } } } private static void configureLogging() { // Set root logger level to DEBUG Configurator.setRootLevel(Level.DEBUG); // Set custom logger levels Configurator.setLevel("SecurityLogger", Level.INFO); Configurator.setLevel("PerformanceLogger", Level.TRACE); } // Simulate user activities and log them private static void simulateUserActivity(Random random) { String[] users = {"Alice", "Bob", "Charlie", "David"}; String[] actions = {"login", "logout", "view_profile", "update_settings"}; String user = users[random.nextInt(users.length)]; String action = actions[random.nextInt(actions.length)]; // Add user and action to thread context ThreadContext.put("user", user); ThreadContext.put("action", action); // Log different user actions with appropriate levels switch (action) { case "login": logger.info("User logged in successfully"); break; case "logout": logger.info("User logged out"); break; case "view_profile": logger.debug("User viewed their profile"); break; case "update_settings": logger.info("User updated their settings"); break; } } // Simulate database operations and log them private static void simulateDatabaseOperations(Random random) { String[] operations = {"select", "insert", "update", "delete"}; String operation = operations[random.nextInt(operations.length)]; long duration = random.nextInt(1000); // Add operation and duration to thread context ThreadContext.put("operation", operation); ThreadContext.put("duration", String.valueOf(duration)); // Log slow database operations as warnings if (duration > 500) { logger.warn("Slow database operation detected"); } else { logger.debug("Database operation completed"); } // Simulate database connection loss with 5% probability if (random.nextInt(20) == 0) { logger.error("Database connection lost", new SQLException("Connection timed out")); } } // Simulate security events and log them private static void simulateSecurityEvents(Random random) { String[] events = {"failed_login", "password_change", "role_change", "suspicious_activity"}; String event = events[random.nextInt(events.length)]; ThreadContext.put("security_event", event); // Log different security events with appropriate levels switch (event) { case "failed_login": securityLogger.warn("Failed login attempt"); break; case "password_change": securityLogger.info("User changed their password"); break; case "role_change": securityLogger.info("User role was modified"); break; case "suspicious_activity": securityLogger.error("Suspicious activity detected", new SecurityException("Potential breach attempt")); break; } } // Simulate performance metrics and log them private static void simulatePerformanceMetrics(Random random) { String[] metrics = {"cpu_usage", "memory_usage", "disk_io", "network_latency"}; String metric = metrics[random.nextInt(metrics.length)]; double value = random.nextDouble() * 100; // Add metric and value to thread context ThreadContext.put("metric", metric); ThreadContext.put("value", String.format("%.2f", value)); // Log high resource usage as warnings if (value > 80) { performanceLogger.warn("High resource usage detected"); } else { performanceLogger.trace("Performance metric recorded"); } } // Custom exception classes for simulating errors private static class SQLException extends Exception { public SQLException(String message) { super(message); } } private static class SecurityException extends Exception { public SecurityException(String message) { super(message); } }}
This class demonstrates the use of different log levels and also shows how to add context to your logs using ThreadContext.
Fluentd is a popular open-source data collector used to forward logs from Log4j to Axiom. The Log4j configuration is already set up to send logs to Fluentd using the Socket appender. Fluentd acts as a unified logging layer, allowing you to collect, process, and forward logs from various sources to different destinations.
To simplify the deployment of the Java app and Fluentd, use Docker. Create a new file named Dockerfile in your project root directory with the following content:
Now that your app is running and sending logs to Axiom, you can view them in the Axiom dashboard. Log in to your Axiom account and go to the dataset you specified in the Fluentd configuration.
Logs appear in real-time, with various log levels and context information added.