在实际工作中会经常遇到一些bug,有些就需要用到文件句柄,文件描述符等概念,比如报错: too many open files, 如果你对相关知识一无所知,那么debug起来将会异常痛苦。在linux操作系统中,文件句柄(包括Socket句柄)、打开文件、文件指针、文件描述符的概念比较绕,而且windows的文件句柄又与此有何关联和区别?这一系列的问题是我们不得不面对的。
ulimit在系统允许的情况下,提供对特定shell可利用的资源的控制。(Provides control over the resources avaliable to the shell and to processes started by it, on systems that allow such control)-H和-S选项设定指定资源的硬限制和软限制。硬限制设定之后不能再添加,而软限制则可以增加到硬限制规定的值。如果-H和-S选项都没有指定,则软限制和硬限制同时设定。限制值可以是指定资源的数值或者hard, soft, unlimited这些特殊值,其中hard代表当前硬限制, soft代表当前软件限制, unlimited代表不限制. 如果不指定限制值, 则打印指定资源的软限制值, 除非指定了-H选项.如果指定了不只一种资源, 则限制名和单位都会在限制值前显示出来.
#<item> can be one of the following: # - core - limits the core file size (KB) # - data - max data size (KB) # - fsize - maximum filesize (KB) # - memlock - max locked-in-memory address space (KB) # - nofile - max number of open file descriptors # - rss - max resident set size (KB) # - stack - max stack size (KB) # - cpu - max CPU time (MIN) # - nproc - max number of processes # - as - address space limit (KB) # - maxlogins - max number of logins for this user # - maxsyslogins - max number of logins on the system # - priority - the priority to run user process with # - locks - max number of file locks the user can hold # - sigpending - max number of pending signals # - msgqueue - max memory used by POSIX message queues (bytes) # - nice - max nice priority allowed to raise to values: [-20, 19] # - rtprio - max realtime priority
该文件指定了可以分配的文件句柄的最大数目(系统全局的可用句柄数目. The value in file-max denotes the maximum number of file handles that the Linux kernel will allocate)。如果用户得到的错误消息审批由于打开文件数已经达到了最大值,从而他们不能打开更多文件,则可能需要增加改之。可将这个值设置成任意多个文件,并且能通过将一个新数字值写入该文件来更改该值。这个参数的默认值和内存大小有关系,可以使用公式:file-max ≈ 内存大小/ 10k.
Historically, the three values in file-nr denoted the number of allocated file handles, the number of allocated but unused file handles, and the maximum number of file handles. Linux 2.6 always reports 0 as the number of free file handles – this is not an error, it just means that the number of allocated file handles exactly matches the number of used file handles.
原因1 - 资源问题:the operating system needs memory to manage each open file, and memory is a limited resource - especially on embedded systems.
原因2 - 安全问题:if there were no limits, a userland software would be able to create files endlessly until the server goes down.
What’s more? If the file descriptors are tcp sockets, etc, then you risk using up a large amount for the socket buffers and other kernel objects, this memory is not going to be swappable.
最主要的是资源问题,为防止某一单一进程打开过多文件描述符而耗尽系统资源,对进程打开文件数做了限制。
6. lsof
lsof(list open files)是一个列出当前系统打开文件的工具。在linux环境下,任何事物都以文件的形式存在,通过文件不仅仅可以访问常规数据,还可以访问网络连接和硬件。所以如TCP和UDP等,系统在后台都为该应用程序分配了一个文件描述符,无论这个文件的本质如何,该文件描述符为应用程序与基础操作系统之间的交互提供了通用接口。因为应用程序打开文件的描述符列表提供了大量关于这个应用程序本身的信息,因此通过lsof工具能够查看这个列表对系统检测以及拍错将是很有帮助的。
/* Define outside of namespace so the C++ is happy. */ struct _IO_FILE;
__BEGIN_NAMESPACE_STD /* The opaque type of streams. This is the definition used elsewhere. */ typedef struct _IO_FILE FILE; __END_NAMESPACE_STD #if defined __USE_LARGEFILE64 || defined __USE_SVID || defined __USE_POSIX \ || defined __USE_BSD || defined __USE_ISOC99 || defined __USE_XOPEN \ || defined __USE_POSIX2 __USING_NAMESPACE_STD(FILE) #endif
#if !defined ____FILE_defined && defined __need___FILE
/* The opaque type of streams. This is the definition used elsewhere. */ typedef struct _IO_FILE __FILE;
<libio.h>
struct _IO_FILE { int _flags; /* High-order word is _IO_MAGIC; rest is flags. */ #define _IO_file_flags _flags
/* The following pointers correspond to the C++ streambuf protocol. */ /* Note: Tk uses the _IO_read_ptr and _IO_read_end fields directly. */ char* _IO_read_ptr; /* Current read pointer */ char* _IO_read_end; /* End of get area. */ char* _IO_read_base; /* Start of putback+get area. */ char* _IO_write_base; /* Start of put area. */ char* _IO_write_ptr; /* Current put pointer. */ char* _IO_write_end; /* End of put area. */ char* _IO_buf_base; /* Start of reserve area. */ char* _IO_buf_end; /* End of reserve area. */ /* The following fields are used to support backing up and undo. */ char *_IO_save_base; /* Pointer to start of non-current get area. */ char *_IO_backup_base; /* Pointer to first valid character of backup area */ char *_IO_save_end; /* Pointer to end of non-current get area. */
struct _IO_marker *_markers;
struct _IO_FILE *_chain;
int _fileno; #if 0 int _blksize; #else int _flags2; #endif _IO_off_t _old_offset; /* This used to be _offset but it's too small. */
#define __HAVE_COLUMN /* temporary */ /* 1+column number of pbase(); 0 is unknown. */ unsigned short _cur_column;