The Linux Programming Interface: A Linux and UNIX System Programming Handbook

You can't have it both ways. if you want to know exactly then you have to drill down to the fundamentals which are handled at a low level (C & assembly). If you want to understand the general concepts you'll need to make lateral moves and study CS along with having enough of the tiny details to fill in the gaps with your intuition.

I think what you may like is a book on the linux API. This is between kernelland and userland.

Try this one:

https://www.amazon.com/Linux-Programming-Interface-System-Handbook/dp/1593272200

Before you do that ensure you know enough basic C. Linux kernel uses K&R style. So this might be good enough and your best bet to fasttrack.

https://www.amazon.com/Programming-Language-2nd-Brian-Kernighan/dp/0131103628

Do keep in mind this is 2K pages worth of reading. This should help to fill in enough of what you don't know you don't know so that you can better use google-fu to get you further.

I feel like C is most useful when you are programming directly to an OS and its resources, rather than through a framework or library. And you don't often need to use the most elegant data structures to accomplish a simple task.

The Linux Programming Interface is still one of the best introductions to Linux programming.

I highly recommend this book: https://www.amazon.com/Linux-Programming-Interface-System-Ha...

All system call interfaces, as well as how system calls happen are thoroughly documented there. Though, I would just use libc, since all this stuff is a bit messy and have plenty of legacy stuff.

Please check "The C Interfaces and Implementations" book by David Hanson [1]. Someone has implemented the concept using his library that you can check and use [2].

Another excellent reference is the "The Linux Programming Interface" book by Michael Kerrisk that documents most of the API available under Linux [3].

[1]https://www.amazon.com/dp/0201498413

[2]https://www.amazon.com/Linux-Programming-Interface-System-Ha...

Assuming this... https://www.amazon.com/Linux-Programming-Interface-System-Ha...

In the way malloc() request memory from heap, there are system calls (for e.g. shmget()) to request/create shared memory segment. If your request is successful, you can copy whatever you like over there. (Yes, you can use memcpy.) But remember to be careful about pointers, a pointer valid for one process, kept in its shared memory, is not necessarily valid for another process using that shared memory segment.

The shared memory is accessible to all processes for reading and/or writing. If multiple processes are reading/writing to a shared memory segment, then, needless to say, some synchronization techniques (for e.g. semaphore) need to be applied.

Please read up on shared memory. An excellent source is "The Linux Programming Interface" by Michael Kerrisk.

Reference:

• http://man7.org/linux/man-pages/man7/shm_overview.7.html
• http://man7.org/linux/man-pages/man2/shmget.2.html
• http://www.amazon.com/The-Linux-Programming-Interface-Handbook/dp/1593272200

Maybe wrong Amazon page (or the store being down made it wonky), I see over 150 reviews. Very stellar response, I'm going to up my backing to get it now:

https://www.amazon.com/Linux-Programming-Interface-System-Ha...

I suggest any of these 3 books (1 is enough to start with; you can get the others later): W Richard Stevens, Stephen A Rago [Advanced Programming in the Unix Environment, 3rd Edn](http://smile.amazon.com/Advanced-Programming-UNIX-Environment-Edition/dp/0321637739) — Marc J Rochkind [Advanced Unix Programming, 2nd Edn](http://smile.amazon.com/Advanced-UNIX-Programming-2nd-Edition/dp/0131411543) — Michael Kerrisk [The Linux Programming Interface: A Linux and Unix System Programming Handbook](http://smile.amazon.com/The-Linux-Programming-Interface-Handbook/dp/1593272200).

I've really enjoyed the book The Linux Programming Interface by Kerrisk as a guide for learning some of these types of low level concepts.

https://www.amazon.com/Linux-Programming-Interface-System-Ha...

I discovered this too for anyone interested. Apparently it is the "New standard" for linux programming.

The Linux Programming Interface: A Linux and UNIX System Programming Handbook

OK, this is what I did. Disclaimer: I'm a hacker in the pure sense of the word and my code ain't the prettiest. I read LDD3 and infiniband source code and other predecessor stuff and decided that "get_user_pages" and pinning them and all that other rigmarole was just too painful to contemplate while hungover. Also, I was working with the other person across the PCIe bus and I was also responsible in "designing" the user space application. I wrote the driver such that at load time, it preallocates as many buffers as it can with the largest size by calling the function myAddr[i] = pci_alloc_consistent(blah,size,&pci_addr[i]) until it fails. (failure -> myAddr[i] is NULL I think, I forget). I was able to allocate around 2.5GB of buffers, each 4MiB in size in my meagre machine which only has 4GiB of memory. The total number of buffers varies depending on when the kernel module is loaded of course. Load the driver at boot time and the most buffers are allocated. Each individual buffer's size maxed out at 4MiB in my system. Not sure why. I catted /proc/buddyinfo to make sure I wasn't doing anything stupid which is of course my usual starting pattern. The driver then proceeds to give the array of pci_addr to the PCIe device along with their sizes. The driver then just sits there waiting for the interrupt storm to begin. Meanwhile in userspace, the application opens the driver, queries the number of allocated buffers(n) and their sizes (using ioctls or reads etc) and then proceeds to call the system call mmap() multiple (n) times. Of course mmap() must be properly implemented in the driver and LDD3 pages 422-423 were handy. Userspace now has n pointers to n areas of driver memory. As the driver is interrupted by the PCIe device, it's told which buffers are "full" or "available" to be sucked dry. The application in turn is pending on a read() or ioctl() to be told which buffers are full of useful data. The tricky part was to manage the userspace to kernel space synchronization such that buffers which are being DMA's into by the PCIe are not also being modified by userspace but that's what we get paid for. I hope this makes sense and I'd be more than happy to be told I'm an idiot but please tell me why. I recommend this book as well by the way: http://www.amazon.com/Linux-Programming-Interface-System-Handbook/dp/1593272200 . I wish I had that book seven years ago when I wrote my first Linux driver. There is another type of trickery possible by adding even more memory and not letting the kernel use it and mmapping on both sides of the userspace/kernelspace divide but the PCI device must also support higher than 32-bit DMA addressing. I haven't tried but I wouldn't be surprised if I'll eventually be forced to.

The Linux Programming Interface is amazing book I am reading now:

http://www.amazon.com/Linux-Programming-Interface-System-Handbook/dp/1593272200

Just look at its outstanding customers range - it is really excellent Linux programming book.

If you want to become a professional and not just a dabbler I would recommend reading some of the following books I have in my bookshelf:

[0] RHCSA & RHCE Training and Exam Preparation Guide by Asghar Ghori. This book will help insure you know your stuff as your system engineer/administrator wise.

[1] A Practical Guide to Linux Commands, Editor and Shell Programming Third Edition. This book will cover the majority of what you would need and want to know when connecting to a remote linux system over ssh.

If you want to get under the hood and become an expert, the following books should help get you started:

[2] Advanced Programming in the UNIX Environment

[3] The Linux Programming Interface: A Linux and UNIX System Programming Handbook

[4] Linux Kernel Development 3rd Edition

To get a nice general overview and get up and going quickly:

[5] How Linux works: What every superuser should know

[6] The Linux Command Line

[7] Python Crash Course

[8] Automate the boring stuff with Python. This is a great book to help you think about how to automate most of the repetitive things you will end up doing on a regular basis.

[0] https://www.amazon.com/RHCSA-RHCE-Red-Enterprise-Linux/dp/14...

[1] https://www.amazon.com/Practical-Guide-Commands-Editors-Prog...

[2] https://www.amazon.com/Advanced-Programming-UNIX-Environment...

[3] https://www.amazon.com/Linux-Programming-Interface-System-Ha...

[4] https://www.amazon.com/Linux-Kernel-Development-Robert-Love/...

[5] https://www.amazon.com/How-Linux-Works-Superuser-Should/dp/1...

[6] https://www.amazon.com/Linux-Command-Line-Complete-Introduct...

[7] https://www.amazon.com/Python-Crash-Course-Hands-Project-Bas...

[8] https://www.amazon.com/Automate-Boring-Stuff-Python-Programm...

OK, if you don't have any real experience in low-level embedded coding (relevant to device drivers), RTOS or OS design in general, file systems, data structures, algorithms, interfaces, etc. And, if you have "hobby level" experience with Assembler, C and C++. And, if your intent is to write a desktop OS, from the ground up, without making use of existing technologies, drivers, file systems, memory management, POSIX, etc. Here's a list of books that could be considered required reading before you can really start to write specifications and code. Pick twenty of these and that might be a good start.

In no particular order:

1- http://www.amazon.com/C-Programming-Language-2nd-Edition/dp/...

2- http://www.amazon.com/The-Answer-Book-Solutions-Programming/...

3- http://www.amazon.com/The-Standard-Library-P-J-Plauger/dp/01...

4- http://www.amazon.com/C-Traps-Pitfalls-Andrew-Koenig/dp/0201...

5- http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp...

6- http://www.amazon.com/Data-Structures-In-Noel-Kalicharan/dp/...

7- http://www.amazon.com/Data-Structures-Using-Aaron-Tenenbaum/...

8- http://www.amazon.com/Mastering-Algorithms-C-Kyle-Loudon/dp/...

9- http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

10- http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

11- http://www.amazon.com/The-Mythical-Man-Month-Engineering-Ann...

12- http://www.amazon.com/The-Programming-Language-4th-Edition/d...

13- http://www.amazon.com/The-Standard-Library-Tutorial-Referenc...

14- http://www.amazon.com/API-Design-C-Martin-Reddy/dp/012385003...

15- http://www.amazon.com/The-Linux-Programming-Interface-Handbo...

16- http://www.amazon.com/Computer-Systems-Programmers-Perspecti...

17- http://www.amazon.com/System-Programming-Unix-Adam-Hoover/dp...

18- http://www.amazon.com/Memory-Programming-Concept-Frantisek-F...

19- http://www.amazon.com/Memory-Management-Implementations-Prog...

20- http://www.amazon.com/UNIX-Filesystems-Evolution-Design-Impl...

21- http://www.amazon.com/PCI-System-Architecture-4th-Edition/dp...

22- http://www.amazon.com/Universal-Serial-System-Architecture-E...

23- http://www.amazon.com/Introduction-PCI-Express-Hardware-Deve...

24- http://www.amazon.com/Serial-Storage-Architecture-Applicatio...

25- http://www.amazon.com/SATA-Storage-Technology-Serial-ATA/dp/...

26- http://www.amazon.com/Beyond-BIOS-Developing-Extensible-Inte...

27- http://www.amazon.com/Professional-Assembly-Language-Program...

28- http://www.amazon.com/Linux-Kernel-Development-3rd-Edition/d...

29- http://www.amazon.com/Version-Control-Git-collaborative-deve...

30- http://www.amazon.com/Embedded-Software-Primer-David-Simon/d...

31- http://www.amazon.com/Programming-Embedded-Systems-C/dp/1565...

32- http://www.amazon.com/Making-Embedded-Systems-Patterns-Softw...

33- http://www.amazon.com/Operating-System-Concepts-Abraham-Silb...

34- http://www.amazon.com/Performance-Preemptive-Multitasking-Mi...

35- http://www.amazon.com/Design-Operating-System-Prentice-Hall-...

36- http://www.amazon.com/Unix-Network-Programming-Sockets-Netwo...

37- http://www.amazon.com/TCP-Illustrated-Volume-Addison-Wesley-...

38- http://www.amazon.com/TCP-IP-Illustrated-Vol-Implementation/...

39- http://www.amazon.com/TCP-Illustrated-Vol-Transactions-Proto...

40- http://www.amazon.com/User-Interface-Design-Programmers-Spol...

41- http://www.amazon.com/Designing-Interfaces-Jenifer-Tidwell/d...

42- http://www.amazon.com/Designing-Interfaces-Jenifer-Tidwell/d...

43- http://www.amazon.com/Programming-POSIX-Threads-David-Butenh...

44- http://www.intel.com/p/en_US/embedded/hwsw/software/hd-gma#d...

45- http://www.intel.com/content/www/us/en/processors/architectu...

46- http://www.intel.com/p/en_US/embedded/hwsw/hardware/core-b75...

47- http://www.hdmi.org/index.aspx

48- http://en.wikipedia.org/wiki/Digital_Visual_Interface

49- http://www.amazon.com/Essential-Device-Drivers-Sreekrishnan-...

50- http://www.amazon.com/Making-Embedded-Systems-Patterns-Softw...

51- http://www.amazon.com/Python-Programming-Introduction-Comput...

52- http://www.amazon.com/Practical-System-Design-Dominic-Giampa...

53- http://www.amazon.com/File-Systems-Structures-Thomas-Harbron...

54- ...well, I'll stop here.

Of course, the equivalent knowledge can be obtained by trial-and-error, which would take longer and might result in costly errors and imperfect design. The greater danger here is that a sole developer, without the feedback and interaction of even a small group of capable and experienced programmers could simply burn a lot of time repeating the mistakes made by those who have already trenched that territory.

If the goal is to write a small RTOS on a small but nicely-featured microcontroller, then the C books and the uC/OS book might be a good shove in the right direction. Things start getting complicated if you need to write such things as a full USB stack, PCIe subsystem, graphics drivers, etc.

The Linux Programming Interface by Kerrisk has pretty thorough coverage of sockets (and everything else).

http://www.amazon.com/The-Linux-Programming-Interface-Handbo...

The Linux Programming Interface: A Linux and UNIX System Programming Handbook

http://www.amazon.com/Linux-Programming-Interface-System-Han...