To enable ELK logging, mostly it will use logback as logging library. However, when migrate existing application, eliminate dependency change is one of the factor to ensure application stability.
In this demo, we will use Log4J2, a common Java logging library to write log to ELK.
Prerequisites
-
Project migrated to gradle;
-
Using Log4J2 as logging library;
Steps
-
Add Spring Cloud Sleuch dependency to project.
In build.gradle, add dependency as below.dependencies { compile 'org.springframework.cloud:spring-cloud-starter-sleuth:2.2.6.RELEASE' }
-
Deploy project to target server.
-
Add Socket appender to write log via Socket call.
In log4j2.xml, add socket appender and enable it.<Configuration status="WARN" monitorInterval="30"> <Properties> <Property name="LOG_PATTERN">%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level --- [%t] %notEmpty{ [%X{x-apigw-requestId}]} - %logger{36} - %msg%n</Property> <Property name="LOG_FILE_NAME">user-management</Property> <Property name="LOG_PATH">c:\log\user-management</Property> </Properties> <Appenders> <Socket name="socket" host="10.3.11.114" port="19722" protocol="TCP"> <JsonLayout properties="true" compact="true" eventEol="true" /> <PatternLayout pattern="${LOG_PATTERN}" /> </Socket> <Async name="async"> <AppenderRef ref="socket" /> </Async> </Appenders> <Loggers> <root level="info"> <AppenderRef ref="async" /> </root> </Loggers> </Configuration>
-
Create Logstash pipeline.
In logstash server, create config file in /etc/logstash/conf.d/ and add content below.
Command## Sample service name: User management service. sudo vi /etc/logstash/conf.d/user-management-service-logstash.conf
Content
## This sample use port 19721 to collect log from socket call in JSON format. input { tcp { port => 19721 codec => json } stdin {} } output { stdout { codec => rubydebug } elasticsearch { action => "index" hosts => [ "10.3.11.114:9200" ] index => "user-management-service-%{+YYYY.MM.dd}" } }
-
Add index pattern in Kibana.
In ELK main menu, click Stack Management > Index Patterns > Create index pattern. -
Configure new index pattern
In Index pattern name, input user-management-service* and un-select include system and hidden indices. After click Net step, select @timestamp in Time field and click Create index pattern.
-
Add index pattern to log.
In ELK main screen, click Logs > Settings
In Log indices, append index pattern created in previous step. -
Test.
In ELK main screen, click Log and select log in filter, if result shown, means integration success.
Leave a Reply