# mmap相关API
Date: 2020-07-22


### 概述

`mmap()`调用进程的虚拟进程空间中一段新的内存映射。

```C
#include <sys/mman.h>
void *mmap(void *addr, size_t length, int prot, int flags,
           int fd, off_t offset);
```

[mmap(2) — Linux manual page](https://man7.org/linux/man-pages/man2/mmap.2.html)

### 用途

| 变更的可见性 | 映射类型 |   |
| --- | --- | --- |
|   | 文件 | 匿名 |
| 私有 | 根据文件内容初始化内存 | 内存分配 |
| 共享 | 内存映射I/O,进程间共享内存(IPC) | 进程间共享内存(IPC) |

### 文件映射

#### 创建步骤

```C
//1. 打开使用的文件
fd = open(argv[1], O_RDONLY);
//2.获取文件信息，文件大小
fstat(fd, &sb);
//3. 生成内存映射
addr = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
```

#### 相关议题

##### 文件权限与内存保护位`prot`

```C
fd = open(argv[1], O_RDONLY);
...
addr = mmap(NULL, sb.st_size, PROT_WRITE, MAP_SHARED, fd, 0);
```

mmap抛出 `errno=13 Permission denied`错误

##### 边界情况

增加一个异常捕获的处理

```C
void signal_handler(int no){

   switch(no){
   case 11:
      printf("get SIGSEGV\n");
      break;
   case 7:
      printf("get SIGBUS\n");
      break;
   default:
      printf("get %d\n", no);
      break;
   }
    exit(-1);
}
```

```C
# ls -l dax.file
-rw-r--r--. 1 root root 7 Jul 18 17:51 dax.file
```

- `SIGSEGV`

访问超过`mmap`映射范围![](/wp-content/uploads/2020/07/mmap1.png)![](/wp-content/uploads/2020/07/mmap2.png)

![](https://garlicspace.com/wp-content/uploads/2020/07/mmap1-1024x546.png)

> 图片来自 The linux programming interface

```C
signal(SIGSEGV, signal_handler);
signal(SIGBUS, signal_handler);
...
addr = mmap(NULL, 6000, PROT_READ, MAP_PRIVATE, fd, 0);
#define ACCESS_OFF  9000
char a = *(addr + ACCESS_OFF);
```

```C
# ./mmaptest dax.file
addr 16ff5000 size 7
get SIGSEGV
```

- `SIGBUS`

访问`mmap`映射的范围大于文件大小按页对齐范围部分。

![](https://garlicspace.com/wp-content/uploads/2020/07/mmap2-1024x555.png)

> 图片来自 The linux programming interface

```C
signal(SIGSEGV, signal_handler);
signal(SIGBUS, signal_handler);
addr = mmap(NULL, 8192, PROT_READ, MAP_PRIVATE, fd, 0);
#define ACCESS_OFF  9000
char a = *(addr + ACCESS_OFF);
```

```C
# ./mmaptest dax.file
addr 35e2000 size 7
get SIGBUS
```

- `Hole Punching`(文件打洞)

对于稀疏文件如果访问`Hole`时mmap是否会报错

![](https://garlicspace.com/wp-content/uploads/2020/07/sparsefile.png)

> This file was made by User:SvenTranslation from wiki [Sparse file](https://en.wikipedia.org/wiki/Sparse_file)

```C
void printf_size(int fd)
{
    struct stat st;
    fstat(fd, &st);
    printf("size %d B(%d B allocate).\n\n",
     st.st_size, st.st_blocks*512);
}

...
fd = open(argv[1],O_RDWR);
fallocate(fd, 0, 0, 4096);
printf_size( fd);

addr = mmap(NULL, 4096*10, PROT_WRITE, MAP_SHARED, fd, 0);

#define ACCESS_OFF  4096*2

fallocate(fd, 0, 4096*5, 4096);
printf_size( fd);

*(addr + ACCESS_OFF) = 'j';

printf_size( fd);

```

```C
# >dax.file
# ./mmaptest dax.file
// 初始化
size 4096 B(4096 B allocate).

# stat dax.file
  File: 'dax.file'
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: fd00h/64768d    Inode: 69436841    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)

// 文件打洞  实际分配 8096字节
size 24576 B(8192 B allocate).

# stat dax.file
  File: 'dax.file'
  Size: 24576           Blocks: 16         IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 69436841    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)

// 写入一个字节, 增加一个IO Block 4096byte
size 24576 B(12288 B allocate).

# stat dax.file
  File: 'dax.file'
  Size: 24576           Blocks: 24         IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 69436841    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)

```

这里可以看到创建带`hole`文件大小 8KB, 对应文件16个 `Blocks`可以看到`stat`结构中`block`\= 512B , 16 _512B = 8KB, 写入一个字符， 增加8个`block` 24_512B = 12KB

```C
# man fstat
struct stat {
   ...
    blkcnt_t  st_blocks;  /* number of 512B blocks allocated */
   ...
};
```

### 匿名映射

可以通过`/dev/zero`可以创建匿名映射, 不用关联具体文件

```C
open("/dev/zero",O_RDWR);
mmap(NULL, sizeof(int), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
...
```

可以通过`MAP_ANONYMOUS`创建匿名映射, fd`位置填写`\-1\`

```C
mmap(NULL, sizeof(int), PROT_READ|PROT_WRITE, MAP_SHARED|
MAP_ANONYMOUS, -1, 0);
```

#### 调整现有映射

`mremap()`，传入要进行扩展内存映射的地址, 并增加`_GNU_SOURCE`的宏定义, `Linux`特有

```C
#define _GNU_SOURCE
...
char *addr = mmap(NULL, sizeof(int), PROT_READ|PROT_WRITE,
 MAP_SHARED|MAP_ANONYMOUS, -1, 0);

char *new_addr = mremap(addr, sizeof(int), sizeof(int)*2,
 MREMAP_MAYMOVE);
```

#### 创建非线性映射

`remap_file_pages()` 可以调整文件分页在内存中出现的顺序, 其目的是为了能够有效地映射大型文件放入有限的32位虚拟地址空间（例如数据库)中.

```C
#define _GNU_SOURCE
...
fd = open("/tmp/tfile", O_CREAT | O_TRUNC | O_RDWR, S_IRUSR | S_IWUSR);

pageSize = sysconf(_SC_PAGESIZE);

addr = mmap(0, 3 * pageSize, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
/* The three pages of the file -- 0 1 2 -- are currently mapped
       linearly. Now we rearrange the mapping to 2 1 0. */
remap_file_pages(addr, pageSize, 0, 2, 0);
remap_file_pages(addr + 2 * pageSize, pageSize, 0, 0, 0);
...
```

在早先版本中,不创建`VMA`结构，后续版本是后续处理和`mmap`类似可以通过`/proc/$PID/maps` 看到新创建的`VMA`, 可以看到调用了两次`remap_file_pages`创建出两个的`VMA`.

```C
# cat /proc/8923/maps
...
7f7f987e5000-7f7f987e6000 rw-s 00002000 fd:00 69436841  /root/src/mmap/dax.file
7f7f987e6000-7f7f987e7000 rw-s 00001000 fd:00 69436841  /root/src/mmap/dax.file
7f7f987e7000-7f7f987e8000 rw-s 00000000 fd:00 69436841  /root/src/mmap/dax.file
```

源码可以看一下:

```C
/*
 * Emulation of deprecated remap_file_pages() syscall.
 */
SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
unsigned long, prot, unsigned long, pgoff, unsigned long, flags)
{
...
   file = get_file(vma->vm_file);
   ret = do_mmap_pgoff(vma->vm_file, start, size,
         prot, flags, pgoff, &populate, NULL); 
         ...
    }
```

`do_mmap_pgoff` -> `do_mmap` 和 `mmap`的 调用链`vm_mmap_pgoff`\->`do_mmap_pgoff`\->`do_mmap`是一致的

### 过度使用内存`overcommit`

进程拥有内存是否可以大于实际可用的标识

```C
0: 需要计算
1: 总是允许
2: 禁止使用
```

看下具体设置

```C
# sysctl -a|grep overcommit
vm.overcommit_memory = 0
vm.overcommit_ratio = 50
```

0， 需要运算

公式: `CommitLimit = (Physical RAM * vm.overcommit_ratio / 100) + Swap`

```C
[root@centosgpt mmap]# cat /proc/meminfo
MemTotal:         481900 kB
...
SwapTotal:       5195316 kB
```

CommitLimit = (481900 \* 50 / 100 ) + 5195316 = 5436266

验证一下,差 2K，`Commited_AS`是已经申请内存大小

```C
# grep -i commit /proc/meminfo
CommitLimit:     5436264 kB
Committed_AS:     853844 kB

```

### `oom`

如果设置了`overcommit`, 当系统内存不能满足现有进程需要时， `oom`出来按照规则清理掉部分进程。

涉及的两个文件,`oom killer`查看进程的 `oom_score`, 分数越大的先杀掉, 用户可以通过调整`oom_adj`(范围 \[-17, +15\])，降低比较中要进程的值，保证正常运行.

```C
/proc/$PID/oom_score
/proc/$PID/oom_adj

```

### `DAX`

针对`NVM`新兴非易失性存储，容量大，断电内容不消失， 目前`Intel`推出的`DCPMM`已经在腾讯云的`Redis`上使用.

可以看下Red hat的[CONFIGURING PERSISTENT MEMORY FOR FILE SYSTEM DIRECT ACCESS](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/configuring-persistent-memory-for-file-system-direct-access-dax)上， 目前是文件系统方式使用

```C
# mkfs -t xfs /dev/pmem0
# mount -o dax /dev/pmem0 /mnt/pmem/
```

### 参考及引用

[用mmap接口访问文件时边界问题会导致的两个错误](http://blog.jcix.top/2018-10-26/mmap_tests/)  
[\[2/2\] mm: replace remap\_file\_pages() syscall with emulation](https://lore.kernel.org/patchwork/patch/462935/)  
[remap\_file\_pages(2) — Linux manual page](https://man7.org/linux/man-pages/man2/remap_file_pages.2.html)  
[Linux 的 OOM Killer 机制分析](http://senlinzhan.github.io/2017/07/03/oom-killer/)  
[理解和配置 Linux 下的 OOM Killer](https://www.vpsee.com/2013/10/how-to-configure-the-linux-oom-killer/)  
[Linux Kernel中AEP的现状和发展](https://cloud.tencent.com/developer/article/1442273)  
[Direct Access for files](https://www.kernel.org/doc/Documentation/filesystems/dax.txt)  
[视频基于英特尔傲腾数据中心级持久内存的 Redis 介绍](https://cloud.tencent.com/edu/learning/course-1583-11540)