性能 – 无法避免在CPU上单独启动的进程上的上下文切换

我正在调查如何在专用CPU上运行进程以避免上下文切换.在我的Ubuntu上,我使用内核参数“isolcpus = 3,7”和“irqaffinity = 0-2,4-6”隔离了两个CPU.我确信它已被正确考虑到：

$cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-4.8.0-27-generic root=UUID=58c66f12-0588-442b-9bb8-1d2dd833efe2 ro quiet splash isolcpus=3,7 irqaffinity=0-2,4-6 vt.handoff=7

重启后,我可以检查一切是否按预期工作.在我运行的第一个控制台上

$stress -c 24
stress: info: [31717] dispatching hogs: 24 cpu, 0 io, 0 vm, 0 hdd

在第二个,使用“顶部”我可以检查我的CPU的使用情况：

top - 18:39:07 up 2 days, 20:48, 18 users,  load average: 23,15, 10,46, 4,53
Tasks: 457 total,  26 running, 431 sleeping,   0 stopped,   0 zombie
%Cpu0  :100,0 us,  0,0 sy,  0,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu1  : 98,7 us,  1,3 sy,  0,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu2  : 99,3 us,  0,7 sy,  0,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu3  :  0,0 us,  0,0 sy,  0,0 ni,100,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu4  : 95,7 us,  4,3 sy,  0,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu5  : 98,0 us,  2,0 sy,  0,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu6  : 98,7 us,  1,3 sy,  0,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu7  :  0,0 us,  0,0 sy,  0,0 ni,100,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
KiB Mem :  7855176 total,   385736 free,  5891280 used,  1578160 buff/cache
KiB Swap: 15624188 total, 10414520 free,  5209668 used.   626872 avail Mem

CPU 3和7是免费的,而其他6个处于完全忙碌状态.精细.

在我的测试的其余部分,我将使用一个几乎纯粹处理的小应用程序

1. It uses two int buffers of the same size
2. It reads one-by-one all the values of the first buffer
   • each value is a random index in the second buffer
3. It reads the value at the index in the second buffer
4. It sums all the values taken from the second buffer
5. It does all the previous steps for bigger and bigger
6. At the end, I print the number of voluntary and involuntary CPU context switches

我正在研究我的应用程序,当我启动它时：

>在非隔离CPU上
>在一个孤立的CPU上

我是通过以下命令行完成的：

$./TestCpuset              ### launch on any non-isolated CPU
$taskset -c 7 ./TestCpuset ### launch on isolated CPU 7

在任何CPU上启动时,上下文切换的数量从20变为……数千

当在隔离的CPU上启动时,上下文切换的数量几乎是恒定的(在10到20之间),即使我并行启动“stress -c 24”.(看起来很正常)

但我的问题是：为什么不是0绝对0？在进程上进行切换时,是为了用另一个进程替换它？但在我的情况下,没有其他过程可以替换！

I have an hypothesis which is that the “isolcpus” option would isolate
CPU form any process (unless the process an CPU affinity would be
given, such as what is done with “taskset”) but not from kernel tasks.
However, I found no documentation about it

我希望任何帮助,以达到0上下文切换

仅供参考,这个问题与我之前开设的另一个问题相关：Cannot allocate exclusively a CPU for my process

这是我正在使用的程序的代码：

#include <limits.h>
#include <iostream>
#include <unistd.h>
#include <sys/time.h>
#include <sys/resource.h>

const unsigned int BUFFER_SIZE = 4096;

using namespace std;


class TimedSumComputer
{

public:
  TimedSumComputer() :
    sum(0),
    bufferSize(0),
    valueBuffer(0),
    indexBuffer(0)
  {}


public:
  virtual ~TimedSumComputer()
  {
    resetBuffers();
  }


public:
  void init(unsigned int bufferSize)
  {
    this->bufferSize = bufferSize;
    resetBuffers();
    initValueBuffer();
    initIndexBuffer();
  }


private:
  void resetBuffers() 
  {
    delete [] valueBuffer;
    delete [] indexBuffer;
    valueBuffer = 0;
    indexBuffer = 0;
  }


  void initValueBuffer()
  {
    valueBuffer = new unsigned int[bufferSize];
    for (unsigned int i = 0 ; i < bufferSize ; i++)
    {
      valueBuffer[i] = randomUint();
    }
  }


  static unsigned int randomUint()
  {
    int value = rand() % UINT_MAX;
    return value;
  }


protected:
  void initIndexBuffer()
  {
    indexBuffer = new unsigned int[bufferSize];
    for (unsigned int i = 0 ; i < bufferSize ; i++)
    {
      indexBuffer[i] = rand() % bufferSize;
    }
  }


public:
  unsigned int getSum() const
  {
    return sum;
  }


  unsigned int computeTimeInMicroSeconds()
  {
    struct timeval startTime, endTime;

    gettimeofday(&startTime, NULL);
    unsigned int sum = computeSum();
    gettimeofday(&endTime, NULL);

    return ((endTime.tv_sec - startTime.tv_sec) * 1000 * 1000) + (endTime.tv_usec - startTime.tv_usec);
  }


  unsigned int computeSum()
  {
    sum = 0;

    for (unsigned int i = 0 ; i < bufferSize ; i++)
    {
      unsigned int index = indexBuffer[i];
      sum += valueBuffer[index];
    }

    return sum;
  }


protected:
  unsigned int sum;
  unsigned int bufferSize;
  unsigned int * valueBuffer;
  unsigned int * indexBuffer;

};



unsigned int runTestForBufferSize(TimedSumComputer & timedComputer, unsigned int bufferSize)
{
  timedComputer.init(bufferSize);

  unsigned int timeInMicroSec = timedComputer.computeTimeInMicroSeconds();
  cout << "bufferSize = " << bufferSize << " - time (in micro-sec) = " << timeInMicroSec << endl;
  return timedComputer.getSum();
}



void runTest(TimedSumComputer & timedComputer)
{
  unsigned int result = 0;

  for (unsigned int i = 1 ; i < 10 ; i++)
  {
    result += runTestForBufferSize(timedComputer, BUFFER_SIZE * i);
  }

  unsigned int factor = 1;
  for (unsigned int i = 2 ; i <= 6 ; i++)
  {
    factor *= 10;
    result += runTestForBufferSize(timedComputer, BUFFER_SIZE * factor);
  }

  cout << "result = " << result << endl;
}



void printPid()
{
  cout << "###############################" << endl;
  cout << "Pid = " << getpid() << endl;
  cout << "###############################" << endl;
}



void printNbContextSwitch()
{
  struct rusage usage;
  getrusage(RUSAGE_THREAD, &usage);
  cout << "Number of voluntary context switch:   " << usage.ru_nvcsw << endl;
  cout << "Number of involuntary context switch: " << usage.ru_nivcsw << endl;
}



int main()
{
  printPid();

  TimedSumComputer timedComputer;
  runTest(timedComputer);

  printNbContextSwitch();

  return 0;
}

今天,我获得了更多关于我的问题的线索
我意识到我必须深入调查内核调度程序中发生的事情.我找到了这两页：

> Ftrace Linux Kernel Tracing
> ftrace – Function Tracer

我在应用程序运行时启用了调度程序跟踪：

# sudo bash
# cd /sys/kernel/debug/tracing
# echo 1 > options/function-trace ; echo function_graph > current_tracer ; echo 1 > tracing_on ; echo 0 > tracing_max_latency ; taskset -c 7 [path-to-my-program]/TestCpuset ; echo 0 > tracing_on
# cat trace

当我的程序在CPU 7(taskset -c 7)上启动时,我必须过滤“跟踪”输出

# grep " 7)" trace

然后我可以搜索从一个进程到另一个进程的转换：

# grep " 7)" trace | grep "=>"
 ...
 7)  TestCpu-4753  =>  kworker-5866 
 7)  kworker-5866  =>  TestCpu-4753 
 7)  TestCpu-4753  =>   watchdo-26  
 7)   watchdo-26   =>  TestCpu-4753 
 7)  TestCpu-4753  =>  kworker-5866 
 7)  kworker-5866  =>  TestCpu-4753 
 7)  TestCpu-4753  =>  kworker-5866 
 7)  kworker-5866  =>  TestCpu-4753 
 7)  TestCpu-4753  =>  kworker-5866 
 7)  kworker-5866  =>  TestCpu-4753 
 ...

答对了！我正在跟踪的上下文切换似乎转换为：

> kworker
>看门狗

我现在必须找到：

>这些进程/线程到底是什么？ (看来它们是由内核处理的)
>我可以避免它们在我的专用CPU上运行吗？

当然,我再次感谢任何帮助:-P