How do you analyze GC logs, thread dumps and head dumps?

-

 Hi awesome community! 

In this article, I would like to describe the one of the toolset (service) for the analyze some problems on different Java-based instances, of course, as Atlassian administrator, first of all, it is Atlassian applications.

First, of the all, I want to say many thanks to the Atlassian Ecosystem developers for the providing Troubleshooting and Support tools. Because a lot of users can provide easy support.zip with thread dumps and GC logs if it is enabled (since 7.4 it is pre-populated in the configs). By the way, you can take a thread dump yourself using the jps ( or old way ps aux | grep java) and  jstack. I hope these useful articles will help you (1, 2, 3, 4, 5).

Well, let's back to our steps about GC, if you click and read the KB about GC logging then you will find already interesting tools GC viewer, Visual GC, but it will be good if we have the same thing in the online and with ML (Machine Learning).

Hence, you can use these online tools which will provide for you easily some view of your system and you can automate it by public REST API. 

* All 3 links work as one.

After uploading you will easier grouping and how threads changed in a different time period.

image.png  

 

For the GC logs, you can see like these parameters

image.png

 

And finally, I like to see a conclusion like this 

image.png  

This article is based on the many questions to me from an online community and AUG members, like how do you review GC logs in short time? 

Hope it helps and you will earn capacity for other awesome tasks.

 

P.S. I hope someone will provide more methods (in atlassian-python-api) for better automate process through support tools instead of bash scripts :)  I'll be happy.

P.S.S.  And we monitoring by prometheus and exporter more detail info you can read in article "Monitoring of Atlassian applications with Prometheus"

P.S.S. Also, you can do it by Zabbix, nagios through JMX.

 

Thanks for your attention.

 

Cheers,

Gonchik Tsymzhitov 

Comments

Post a Comment

Popular posts from this blog

Overview and practical use cases with open source tracing tool for Java Apps. Glowroot. Installation

-
Image
In that small article, you can find how to install the agent of Glowroot. Installation of the application is quite simple on agents, and this article shows a scheme for working with a built-in collector, but for industrial installations we use it with a bundle with elasticsearch. The installation can be both for individual nodes and so for the central collector. In this case, consider for a separate node. Prerequisites: {jira_home} - is location of Jira home directory. {jira_installation_directory} is the location of Jira installation directory.   Steps: Downloading the installer wget -c https://github.com/glowroot/glowroot/releases/download/v0.13.6/glowroot-0.13.6-dist.zip To check, create a directory via command   mkdir -p /{jira_home}/glowroot/tmp Let the Jira process write and set ownership for simplicity, but it is recommended that you just give the write ability to the owner of the Jira process. chown -R jira: /{jira_home}/glowroot We additionally set the argument to setenv.sh in
Read more

Atlassian Community, let's collaborate and provide stats to vendors about our SQL index usage

-
  Hi!  Gonchik is in touch. Today I would like to get a little bit of help from you related SQL indexes on your Jira installation (Data Center, Server edition does not matter).    I would like to share a story about  PostgreSQL  and  Jira 8.13.3  ( JSD 4.13.3 )  release.  In that post I will be happy if you answer and comment about your instance indexes, as those info can help to improve the existing situation of heavy indexes.  Typically, it starts from improvement of configuration (e.g. using  https://postgresqltuner.pl/ ),  then query analyzing, adding indexes, after removing as heavy one (shortly  https://wiki.postgresql.org/wiki/Index_Maintenance ) :)    I use like this query in PostgreSQL: SELECT s.schemaname,        s.relname AS tablename,        s.indexrelname AS indexname,        pg_size_pretty(pg_relation_size(s.indexrelid::regclass)) AS index_size,        idx_tup_read,         idx_tup_fetch,         idx_scan FROM pg_catalog.pg_stat_user_indexes s JOIN pg_catalog.pg_index i O
Read more

How only 2 parameters of PostgreSQL reduced anomaly of Jira Data Center nodes

-
Image
  Hello community, Gonchik Tsymzhitov is in touch. Today, I would like to share with you a short story about postgresqltuner.pl in usage.  So from previous articles I hope you configured the PostgreSQL server and postgresqltuner.pl.  After executing command, ./postgresqltuner.pl --ssd for ssd disks server. You can see like this output: postgresqltuner.pl version 1.0.1 [OK]      I can invoke executables Connecting to /var/run/postgresql:5432 database template1 as user 'postgres'... [OK]      The user account used by me for reporting has superuser rights on this PostgreSQL instance =====  OS information  ===== [INFO]    OS: linux Version: 5.4.17-2036.102.0.2.el8uek.x86_64 Arch: x86_64-linux-thread-multi [INFO]    OS total memory: 41.03 GB [OK]      vm.overcommit_memory is adequate: no memory overcommitment [INFO]    Running under a vmware hypervisor [INFO]    Currently used I/O scheduler(s): mq-deadline =====  General instance informations  ===== -----  PostgreSQL version  -----
Read more