Virtualization and Xen

In the finale of 2015 and considering the overall absence of articles on my blog this year, I thought of adding another piece of writing so that I don’t call this year a complete failure.

This is my attempt to easily explain virtualization concepts and Xen, including a glimpse into manual Xen hypervisor and guest installation on Debian 8. I’ll also briefly mention cloud computing and containers since these are related concepts, and modern developments. Have some coffee, this will be a long one.

Read more…

Vim: a text-editing gamepad

Although Vim has been my editor of choice for years now, I am nowhere near being a power user. That’s simply because I didn’t do any real, full-time software development, and for that reason I didn’t have the need for any advanced features. Moreover, a lot of the power of Vim is hidden if you are just a casual user. On my new role I will be spending some more time coding, so I wanted to explore Vim a bit more and pick up some extras.

This article provides a brief overview of Vim 7, some useful key strokes and configuration settings, links to essential resources, with added focus on Python integration.

Read more…

Sys V init –status-all

How did I not know about service –status-all before, which is used to check the status of all services controlled by the init system?

Network packet drop locations

Let’s look at the network transmit and receive paths between two TCP endpoints, and understand at which places in the Linux network stack can we expect packet blocking or packet drops.

Read more…

PSS: proportional set size

While looking at /proc/PID/smaps to view memory consumption of the process’s mappings, you might be tempted to look at the RSS (resident set size, the amount of memory resident in RAM), and use that as ultimate, realistic measure of memory usage of a process. The problem with RSS however is that although it accounts for memory of shared libraries (as long as they are in RAM), we don’t have super accurate information on shared pages. For example, you can have 100s of httpd processes, and summing up their RSS would overestimate the total memory usage.

On the other hand, PSS (proportional set size) of the process treats shared memory as a proportion used by the current process. It charges a given process for shared pages divided by the number of sharers. As explained by the LWN article, if a process has 1000 pages all to itself, and 1000 shared with one other process, its PSS will be 1500.

So, when you plot your processes for memory usage, plot both RSS and PSS. Also, when working in a shell you can sum ‘Pss:’ for each mapping of the process:

$ cat /proc/6440/smaps  | grep ^Pss: | awk '{s += $2} END { print s }'

NSS vs. dig vs. telnet

One of my colleagues today had a customer who suffered a database failover. The failover workflow includes an automatic change to the database’s DNS A record, to point to the new host’s IP. The database A record has a TTL of 5 seconds. However, the customer’s application couldn’t connect to the database. Performing a dig against the database hostname returned the new IP address, however, telnet returned a different IP address.

Read more…

Memory fragmentation troubleshooting

This article provides elementary overview of the buddy allocation algorithm, memory fragmentation issues it introduces, and how to obtain some more insight on this in Linux.

Read more…

More with less

Here are a few tricks I use every day with GNU less.

Read more…

TCP/IP offloading and per-packet optimization

Overhead of network stack processing in high-speed networks (1/10 GbE) is significant, to the point where it becomes the bottleneck. CPU cannot keep up with the busy I/O. The technology to entirely free the CPU from network processing tasks directly to the NIC itself is called TCP offload engine (TOE). If TOE is not supported by the NIC, there are alternatives with the help of OS to offload some of these operations.

Read more…

MTU

MTU is the maximum frame size that can be transmitted on a network link without fragmentation. It is associated with the NIC, and the link layer protocol.

Read more…