Posts

Showing posts with the label latency

Stories about detecting Atlassian Confluence bottlenecks with APM tool [part 1]

Image
  Hey! Gonchik, a lover of APM (application performance monitoring) tools, in particular Glowroot, is in touch. Today I will tell you how to find bottlenecks in Confluence On-Prem in the shortest possible time based on one industrial installation. We are faced with a situation where a large number of people simultaneously break into the knowledge base on Confluence On-Prem (during the certification), and the Confluence dies for some time. We immediately thought that the problem is precisely in the simultaneous number of visitors and we can immediately tweak the JVM, but it turned out that not everything is so simple. Below I will tell you how we found the real cause of the brakes and how we dealt with it. The main task: to conduct an audit and, on its basis, achieve performance improvements, especially in times of a large number of active users in the system. Of course, first of all, the hardware resources and OS configurations were checked, where no problems were identified. Howev...

Истории о том, как с помощью APM инструмента найти узкие места в Atlassian Confluence

Image
  Привет!  На связи Гончик, любитель APM (application performance monitoring) инструментов, в частности Glowroot .  Сегодня расскажу о том, как за кратчайшее время найти узкие места в Confluence On-Prem на основе одной промышленной инсталляции.  Мы столкнулись с ситуацией, когда большое количество людей одновременно ломится в базу знаний на Confluence On-Prem (во время сдачи аттестации), и конфлюенс умирает на какое-то время. Сразу подумали, что проблема именно в одновременном количестве посетителей и сразу можно затюнить JVM, но оказалось, что не все так однозначно. Ниже расскажу, как мы нашли реальную причину тормозов и как с ней справились. Основная задача: провести аудит и на его основе добиться улучшения производительности, особенно в моменты большого количества активных пользователей в системе.  Конечно, в первую очередь были проверены ап паратные ресурсы, конфигурации ОС, где никаких проблем не выявлено.  Однако, отсутствовали access логи nginx, поск...

Overview and practical use cases with open source tracing tool for Java Apps. Glowroot. Installation

Image
In that small article, you can find how to install the agent of Glowroot. Installation of the application is quite simple on agents, and this article shows a scheme for working with a built-in collector, but for industrial installations we use it with a bundle with elasticsearch. The installation can be both for individual nodes and so for the central collector. In this case, consider for a separate node. Prerequisites: {jira_home} - is location of Jira home directory. {jira_installation_directory} is the location of Jira installation directory.   Steps: Downloading the installer wget -c https://github.com/glowroot/glowroot/releases/download/v0.13.6/glowroot-0.13.6-dist.zip To check, create a directory via command   mkdir -p /{jira_home}/glowroot/tmp Let the Jira process write and set ownership for simplicity, but it is recommended that you just give the write ability to the owner of the Jira process. chown -R jira: /{jira_home}/glowroot We additionally set the argument to set...

How to investigate the indexing time of custom fields in Jira Data Center installation?

Image
     In that article, I would like to share the usage of new functionality from 8.10 Jira DC in the cluster monitoring tab and say thank you to the Atlassian development team. Because previous we do parsing logs, check indexes via Luke, and do testing via benchmark tools.      As we know many recommendation around custom fields creation:  Government (documentation and do more generic etc) Context configurying (https://confluence.atlassian.com/adminjiraserver/custom-fields-with-global-contexts-956713282.html) Standard indexers vs custom indexers Periodically do a cleanup (with custom app, with built-in functionality) Custom field optimizer But one problem is unclear, when we need to start to investigate a problem, why is SysOps annoying if we want to add new custom fields, how to measure?       So Atlassian enrolled in 8.10.x the UI and released a feature with the name the more insights for custom fields. It’s part of feature Ind...

How to check the fragmentation and do defragmentation on ext4? Helpful check for home directory of application

Image
     Today, I got a question about how to check the fragmentation of ext4 filesystems, as most common for the Data Center edition or Server edition setups.       That article is the next step of a previous story about the  improvement of latency for volumes.      Of course, some users can ask why we need to do defragmentation on ext4 as it’s not fragmented. As an answer, I would recommend checking out the presentation from LinuxCon 2010   with the name Outline of Ext4 File System & Ext4 Online Defragmentation Foresight   and watch the small video with fragmentation So let’s start to use one small command line utility e4defrag.  Online man page you can find here: http://manpages.ubuntu.com/manpages/focal/man8/e4defrag.8.html Get report of fragmentation:      $ sudo e4defrag -c /location After checking the results you can run the defragmentation: $ sudo e4defrag /location If you want extra info: $...