Logic contention s detected on net

Опять здравствуйте. Вчера я захотел к своему проекту прифигачить дисплей, он на 44780, к ATmega8. Следуя инструкциям я перепробовал много вариантов, которые находил в инете, но т.к. реализовать это хотелось самому — не во всех мог легко разобраться. Да и волновал то меня только принцип работы. Сделать я это решил на 8-битной шине т.к. проще.

В официальной документации я вычитал, что если дать дисплею время на запуск, то он дефолтно выставит параметры работы. Так я и сделал. К тому же всему проверяю Busy flag. Но почему-то желанная буква так и не появляется на экране, а протеус вываливает в лог 7 сообщений "logic contention(s)". В чем же проблема? Спасибо.

Прикрепил прошивку, проект для протеуса и доки.

More CPUs but less throughput?

Thread contentions can be tricky to find with basic monitoring, especially if your application reaches a maximum throughput but you don’t receive an alert from your monitoring system.

Furthermore, you may face this problem only when deploying your application in production because concurrency problems are more visible on a multi-core system.

This is another good reason to always perform a benchmark campaign on the target architecture.

Thread contention symptoms

While your application is under a maximum load you can observe that:

• Your application is not fully CPU bound. i.e. the percentage of time that the CPUs were idle is more than 10%
• There are no obvious limitations (i.e: you can check that all of the following items are true):
• the percent of CPU iowait is low.
• the disk and network bandwidth usage is fine,
• the throughput of the garbage collector is good
• there are no database bottleneck
• the datasources pool size is larger than the number of running threads

Although it may appear to be significant, in this situation, the number of context switches per second is not relevant.

It’s possible that someone has put some Thread.sleep in the application.

Detecting the thread contention

The easiest way to see contentions is by using VisualVM when the application is under load and looking at the Threads tab. In the worst case you will see something like this:

VisualVM threads timeline

The red line shows the threads that are either in a monitor region or waiting in an entry set for the monitor.

The monitor is the mechanism that Java uses to support synchronization. You can find more information about the monitor here.

Analyzing

Now click on the Thread Dumps button and look at the BLOCKED threads, you should find many BLOCKED threads waiting for the same locks:

VisualVM threads dump

In this case threads are locked on the StreamHandler.publish method:

This syncrhonized instruction becomes a problem on high concurrency.

Automating thread contention detection

VisualVM is doing a great job here but what if you want to have a metric of the thread contention in your continuous integration benchmark job?

The JMX Threading MBean ( java.lang:type=Threading ) exposes thread contention statistics. First you need to enable it by setting the property ThreadContentionMonitoringEnabled to True. Then invoking the dumpAllThreads method of the same mbean will return a list of thread objects with the number of times that each thread has been blocked and the amount of time they were waiting in a monitor state.

After having a hard time getting jmxsh to do this for me, I started writing a command line tool that can work like iostat, vmstat or sar, something like a jmxstat . Thanks to the social coding, GitHub already owns a jmxstat project done by Jonathan Lee.

One fork later I got something like this:

The —contention option adds the blockedCount and blockedTimeMs columns. These are the sum of blocked count and time for all threads.

In addition jmxstat can trace any mbean attribute and performs a Full Garbage Collection without having to install an X server! Visit the GitHub page for more information about jmxstat.

Here is an example of contention monitoring during a benchmark with a rampup from one to ten threads generated from the jmxstat output

Blocked count and times chart

I wasn’t able to measure the tool overhead on our benchmarks but of course it may depend on the tested application so it must be checked first.

This metric is now part of our daily bench monitoring report.

Footnotes:

For information about the Nuxeo Platform, please visit our product page or request a custom demo to see how we can help your organization.*

This kind of chart can be easily generated using tcsv2png

Can someone please explain simply what thread contention is?

I have googled it, but cannot seem to find a simple explanation.

10 Answers 10

Essentially thread contention is a condition where one thread is waiting for a lock/object that is currently being held by another thread. Therefore, this waiting thread cannot use that object until the other thread has unlocked that particular object.

Several answers seem to focus on lock contention, but locks are not the only resources on which contention can be experienced. Contention is simply when two threads try to access either the same resource or related resources in such a way that at least one of the contending threads runs more slowly than it would if the other thread(s) were not running.

The most obvious example of contention is on a lock. If thread A has a lock and thread B wants to acquire that same lock, thread B will have to wait until thread A releases the lock.

Now, this is platform-specific, but the thread may experience slowdowns even if it never has to wait for the other thread to release the lock! This is because a lock protects some kind of data, and the data itself will often be contended as well.

For example, consider a thread that acquires a lock, modifies an object, then releases the lock and does some other things. If two threads are doing this, even if they never fight for the lock, the threads may run much slower than they would if only one thread was running.

Why? Say each thread is running on its own core on a modern x86 CPU and the cores don’t share an L2 cache. With just one thread, the object may remain in the L2 cache most of the time. With both threads running, each time one thread modifies the object, the other thread will find the data is not in its L2 cache because the other CPU invalidated the cache line. On a Pentium D, for example, this will cause the code to run at FSB speed, which is much less than L2 cache speed.

Since contention can occur even if the lock doesn’t itself get contended for, contention can also occur when there is no lock. For example, say your CPU supports an atomic increment of a 32-bit variable. If one thread keeps incrementing and decrementing a variable, the variable will be hot in the cache much of the time. If two threads do it, their caches will contend for ownership of the memory holding that variable, and many accesses will be slower as the cache coherency protocol operates to secure each core ownership of the cache line.

Ironically, locks typically reduce contention. Why? Because without a lock, two threads could operate on the same object or collection and cause lots of contention (for example, there are lock free queues). Locks will tend to deschedule contending threads, allowing non-contending threads to run instead. If thread A holds a lock and thread B wants that same lock, the implementation can run thread C instead. If thread C doesn’t need that lock, then future contention between threads A and B can be avoided for awhile. (Of course, this assumes there are other threads that could run. It won’t help if the only way the system as a whole can make useful progress is by running threads that contend.)