Log Request Body in Kong Ingress Logs

I am Currently working on the kong ingress and there is requirement came to me, in which we need to log the request body in the kong-ingress logs, so i have find out the way by which you can able to achieve this and log the request body in the kong-ingress logs.

Let’s Start Guys:-

So to get the request body logged in the kong logs, we first need to change the log_format of the kong-ingress.

If you are using the helm-chart, we just need to add one environmental variable in the proxy container, that is NGINX_HTTP_LOG_FORMAT, by default the log-format is different and will be something like

#Default log Format

$remote_addr — $remote_user [$time_local] “$request” $status $body_bytes_sent “$http_referer” “$http_user_agent”

to change this simply add the env variable mentioned above that is nginx_http_log_format, now to get the request body, you need to add the $request_body field added in the kong logs format.

#Modified Log Format

NGINX_HTTP_LOG_FORMAT: show_everything $remote_addr — $remote_user [$time_local] “$request” $status $body_bytes_sent “$http_referer” “$http_user_agent” “$request_body”

now just apply the helm-chart with the new values and you will see the request body start getting logged in the proxy container logs.

Testing:-

curl — location ‘https://cm-wh-xxxxx.domain.com/health' — header ‘content-type: application/json’ — data ‘{“test-string”: “Hello-World”}’

Output:-

10.200.2.90 — — [ 25/May/2024:10:06:07 +0000]POST /health HTTP/1.1 200 11 https://cm-wh-xxxxx.domain.com/health/ PostmanRuntime/7.37.3 — — {\x22test-string\x22: \x22Hello-World\x22}

As you can see the request body is start getting logged in the ingress logs at the end.

But this looks quite simple, and if it is not working for you, you can check out this blog, really helpful

https://tech.aufomm.com/how-to-log-request-and-response-body-with-kong/

Scenario:- But There is a one more requirement in this in which we only need to log the request body when the status code is 4xx. How to Do it, i tried it using the pre-function and post-function plugin available in the kong, but there is no luck, might be i am doing something wrong, But there is another way by which we can achieve this and can we helpful in doing other stuff as well in the kong, using the custom nginx template, read about this more on:- https://docs.konghq.com/kubernetes-ingress-controller/latest/reference/faq/nginx.conf/

To do this conditional logging, only log the request body when 4xx, here are steps:-

1. Create a custom nginx template in which this conditional logging logic is present.
2. Now to do this, you need to take the help of the map in the nginx template to achieve this requirement.
3. Full file is at location:-https://gist.github.com/tarun9715m/4702be9d8b252ffe4e0342484baba700

#START
#This is Added to not add request_body in logs, when status is not 400, This block is the only-change in this
#file rest all fields are same 2 same as of the original file, exists at the location:- /usr/local/share/lua/5.1/kong/templates/nginx_kong.lua
#inside the proxy container

map "$status" $request_body_log {
  "~^[4]" "$request_body";
  default "Request Body is only available when status_code is 4xx";
}
#END

Here you can see I used the keyword $request_body_log this is the custom variable which derived it value from this map, for e.g if 4xx status code is there then value becomes $request_body, and if there is non 4xx status code then it will become some string like [Request Body is only available when status_code is 4xx].

3. Now just follow the steps mentioned in the above mentioned kong website url.

4. Create the config map, mount the config map

5. Now as we have did this custom conditional logging and saving the value in the variable $request_body_log, we need to put this variable in the log format.

6. Now my log-format will become

NGINX_HTTP_LOG_FORMAT: show_everything $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$request_body_log"

Now let’s check putting the same curl again

curl - location 'https://cm-wh-xxxxx.domain.com/health' - header 'content-type: application/json' - data '{"test-string": "Hello-World"}'

Output

10.200.2.90 - -[25/May/2024:10:37:10 +0000] POST /health HTTP/1.1 200 11 https://cm-wh-xxxxx.domain.com/ PostmanRuntime/7.37.3 --- Request Body is only available when status_code is 4xx 

You can clearly see the status is 200 and that’s why they have added the string “Request Body is only available when status_code is 4xx”

Now let’s put something different path which is not available that will give 4xx status code and the request body will get logged.

curl - location 'https://cm-wh-xxxxx.domain.com/health1111' - header 'content-type: application/json' - data '{"test-string": "Hello-World"}'

Output:-

10.200.2.90 - -[25/May/2024:10:40:12 +0000]POST /health11111 HTTP/1.1 401 40 https://cm-wh-xxxxx.domain.com/  PostmanRuntime/7.37.3 --- {\x22test-string\x22: \x22Hello-World\x22} 

That’s it, our aim for getting the request body logged when the status code is 4xx is solved.

I Hope might be this is not useful for you or you have some different use-case to cater but this custom nginx template implementation can help you somehow in future.

Thanks for reading it guys. That’s it for this article on the kong. See you soon in any other blog and I think this guide will help you to achieve the same setup and learn something new about the kong-ingress conditional logging and how to use the custom-nginx-template.

In Case of Any Issues and Questions and Suggestion, you can connect me on LinkedIn:

https://www.linkedin.com/in/tarun-mittal-1507/