Determine the up time of an TCP connection on Linux

As a typical use case for troubleshooting, we may need to find out the up time of an established socket connection. Well, how can we do that inside a running Linux system? I faced this issue this afternoon, and got some luck finally. Note that it’s limited to TCP and UDP.

This is doable by using the info from inode with some scripting. Firstly we can find the inode info from /proc/self/net/tcp and /proc/self/net/udp, then check out the running time from /proc/PID/fd. As sockets info in /proc/self/net/tcp are hexadecimal numbers and “encoded”, here is a ruby script for that,

#!ruby

def unpack_raw(addr)
  # extract the network information from the raw ip:port in hex.
  raw_ip, port = addr.split(/:/).map(&:hex) # convert the hex values to integers
  ip = ""
  4.times do
    ip += (raw_ip & 0xFF).to_s # pull the next octet
    ip += "." # append a dot
    raw_ip = raw_ip >> 8 # shift down the octets
  end
  return ip.chop, port # String#chop is used because we'll have a trailing dot.
end

NETWORK_FILES = %w[tcp udp]

inode_hash = { } # mapping of inode -> fd file

Dir["/proc/*/fd/*"].each do |fd|
  link = (File.readlink(fd) rescue nil)
  if link and link =~ /\Asocket:\[/
    inode_hash[link.match(/\Asocket:\[(.*?)\]\z/)[1]] = fd
  end
end

NETWORK_FILES.each do |file|
  lines = File.readlines("/proc/self/net/#{file}")
  lines.shift
  network_info = lines.map { |line| line.split(/\s+/) }
  network_info.each do |info|
    begin # file descriptors can vanish at any time; just bail if we can't get the info
      local, remote = info[2..3]
      local_str     = unpack_raw(local).join(":")
      remote_str    = unpack_raw(remote).join(":")

      inode       = info[10] # get the inode from the text
      fd_file     = inode_hash[inode] # look up our fd file
      owner       = File.stat(fd_file).uid # get the owner of the fd
      pid         = fd_file.match(%r!/proc/(.*?)/!)[1] # get the pid of the fd
      executable  = File.readlink("/proc/#{pid}/exe") # get the process the fd belongs to

      # this prints our output
      puts "#{file}: local:#{local_str} remote:#{remote_str} inode:#{inode} pid:#{pid} uid:#{owner} exec:#{executable}"
    rescue
    end
  end
end

Copy and name it as “socket-info.rb”, then run it like this,

# ruby  socket-info.rb
tcp: local:0.0.0.0:29090 remote:0.0.0.0:0 inode:628775136 pid:19741 uid:99 exec:/opt/haproxy-1.5.12/sbin/haproxy
tcp: local:0.0.0.0:10050 remote:0.0.0.0:0 inode:179730999 pid:31169 uid:998 exec:/usr/sbin/zabbix_agentd
tcp: local:0.0.0.0:29091 remote:0.0.0.0:0 inode:628775138 pid:19741 uid:99 exec:/opt/haproxy-1.5.12/sbin/haproxy
tcp: local:0.0.0.0:10820 remote:0.0.0.0:0 inode:30550679 pid:6618 uid:1000 exec:/opt/jdk1.8.0_45/bin/java
tcp: local:0.0.0.0:29093 remote:0.0.0.0:0 inode:628775143 pid:19741 uid:99 exec:/opt/haproxy-1.5.12/sbin/haproxy
tcp: local:0.0.0.0:29097 remote:0.0.0.0:0 inode:628775144 pid:19741 uid:99 exec:/opt/haproxy-1.5.12/sbin/haproxy
tcp: local:0.0.0.0:10826 remote:0.0.0.0:0 inode:652328993 pid:5584 uid:1000 exec:/opt/jdk1.8.0_45/bin/java
...

Now we have the inode info for each socket, and all the next to do is to retrieval inode details from each running process accordingly, as this,

# find /proc/19741/fd -lname "socket:\[628775136\]" -printf %t
Tue Feb 23 15:08:35.0486045847 2016

Here the PID is 19741, which is the process ID of Haproxy as listed above, and 628775136 the inode number for a tcp connection.

Please feel free to leave a comment below if you have any queries.

Share Button

One thought on “Determine the up time of an TCP connection on Linux

Leave a comment

Your email address will not be published. Required fields are marked *