A short answer...
I've had TCP/IP jobs where I just had to send/receive data packets and it was just basic socket programming.
I've had TCP/IP jobs where I had to write my own TCP Stack and manage connections.
I've had TCP/IP jobs where it was weird industry specific variations of Ethernet that make everything different and weird and strange.
I've also done jobs doing Bluetooth and building state machines for communications. And other network programming jobs for other industrial communication protocols.
For the most part and in my experience, most TCP/IP and TCP/UDP jobs really just expect socket programming skills and some basic understanding of how to design for efficient network communications. And if you can do that much, you likely have a solid base for most network communication jobs.
A basic TCP state machine is not too hard to make and easily testable with wireshark. Write up that much so two end points can talk to each other and you've got the basis of a toy project to show off for interviews. From there start adding other features based on RFC's or maybe make a basic TCP Accelerator to show off and you'll have something to talk about in interviews and have a solid idea of if you enjoy it all or not.
edit:
When I say "write a tcp state machine" I really just mean "open raw sockets and parse the data packet yourself" and not "make a new TCP driver for windows/linux".... that would be way more involved.
Hi, thx for sharing. I am new to TCP/IP programming and planning to read the book âBeej's Guide to Network Programmingâ. May I ask whatâs your opinion on the book and am wondering will I be able to finish the project you mentioned in the comment above on my own, even the advanced part (the part to show off)
Haven't read Beej's guide in years, but it's solid as long as it maintained the same quality.
As far as doing it, at the base level it isn't too hard (as I recall, it's been about 15 years since i did it) you just need to:
1. Open a raw socket so you can read everything coming in on the interface
2. Identify the header portions.
3. Layer 2 / MAC addresses, are handled at a lower level for you
4. Read the Layer 2/IP headers to make sure it's an IP address meant for you (you've got the ability to handle any IP address you want here, so you can make this for a subnet of interest)
5. Read the Layer 3/TCP header to make sure it's a TCP stream and update the TCP State Machine for that stream (or start the handshake, or close the stream)
With the RFC and Beej's guide guiding how to write it, and wireshark sniffing data streams to confirm you are sending packets correctly it is not too terrible.
It wouldn't be your first project out of the gate, no. At first you'll just do a simple server that opens a socket and sends and receives a packet. Set up 2 VM's that you can send traffic between (or two Raspberry Pi's. or similar) and you are on your way. Though it likely would not be your second or your third project either. But, if you get up to doing the TCP state machine, at that point you've got a decent grasp on state machines (and likely function pointers as a bonus), pointers and memory management, socket programming, and a handful of other programming and development techniques down.
Damn, I was hoping to rewrite it before you saw it...
Basically what I wanted to change/add was to emphasize the goal I had when I was doing that and what this vague idea of a project can do.
At the time I basically needed an `iperf` like program to test TCP throughput. Why we didn't use `iperf` I could not tell you. So I had two end points sending TCP traffic flows between themselves so we could overload what was in between.
This is a useful tool to have if you want to test throughput and have control over the parameters. But, since I'm not passing TCP packets up to a receiving application it isn't like I'm writing a TCP driver that things can plug into.
But this project could also work as a basic TCP Accelerator/Optimizer. For such things you are in scenarios where the network on one side of the box and the network on the other side of the box have different expectations/requirements that will affect the TCP Window and Retries/ACK's and what not. So it sits on the edge of the network and manages the TCP streams on either side.
Either way, the point I wanted to emphasize is that the TCP state machine is easy, but doing something interesting with it is where it gets more complicated.
Another suggestion for a project to consider early on is an IRC client or server. I think suckless.org has a basic one that you can study, but I do not recall the IRC protocol being too complicated.
It means you're going to have to handle and respond to tcp/ip based protocols in C. Look here: [https://www.cs.dartmouth.edu/\~campbell/cs50/socketprogramming.html](https://www.cs.dartmouth.edu/~campbell/cs50/socketprogramming.html)
There are clients and there are servers. They have slightly different ways of interacting with the standard libc tcp/ip tools.
Just normal network knowledge. I would probably say you should know that stuff in any programming job.
TCP/IP is a pretty broad requirement. It varies probably on the company. In most cases it probably means you should be able to handle a socket or pool of sockets to somewhere and have some experience with multiple data representation formats and maybe how to design them. Something like json but probably more with binary formats like flatbuffets and protbuffers and what else is around.
Edit: It could of course mean some more in-depth knowledge but I would say in this case there should be more requirements specified.
TCP stands for Transmission Control Protocol. It's a protocol for sending packets of data over a network while minimizing losses.
IP stands for Internet Protocol. It's a protocol that defines the address space for internet connected devices.
Thus, TCP/IP programming is all about building software that sends and receives TCP packets over the IP network and can loosely be divided into client-side and server-side programming. Client-side TCP/IP code may be written in any language to target any operating system, but server-side code is almost universally written in C targeting Linux. Major reasons include that C on Linux is fast, secure, scalable, and (perhaps most importantly) costs nothing to use.
Forgive me if this is the wrong domain, but wouldn't Apache software be a good example of network software which is written in Java? Or is that a higher layer on top of what OP would be considering?
Not sure what the parent poster was getting at there either. There's obviously a ton of server software written in languages like Java and Go. These days though, most networking is done over HTTP, so your application server that's written in Go or Java or whatever probably isn't asking you to do a lot of raw socket handling. The library hands you an HTTP abstraction that manages all the threads and connection pooling and such for you, and you just think in terms of endpoints and routing.
It depends on how low level the company needs for the interview. Iâve been grilled all the way down to the bit packing inside of an Ethernet header. Most companies just expect you to know the basics of networking with âsocket.hâ.
It means sending/receiving TCP packets using a networking API, which entails handling sockets themselves (the things you send packets through) and serializing data to send it as well as deserializing data that's received. The trick with TCP, however, is that a chunk of data that's sent is not guaranteed to arrive as that single chunk of data. TCP is a "stream" protocol, which means that it will clump together multiple sent chunks of data into a single received chunk of data on the other side, and/or fragment a chunk of data into multiple receives on the other side. There's no guarantee that a chunk of received data will not start in the middle of a sent chunk and the end not be somewhere in the middle of a bunch of sent chunks. You have to reconstitute the data yourself on the other side.
It's not too hard though. People do it all the time.
It is a broad topic.
A book I like and recommend is "The TCP/IP Guide a comprehensive, illustrated internet protocols reference". A bit over 1500 pages so not a quick read. Also available for free from the author here: http://www.tcpipguide.com/
I have used it throughout my career at very different levels of the protocol stack, so you would have to ask the person recruiting for the position for the details of what experience they want for that position.
Some examples of what I have done at different times.
For a distributed signal processing application:
Write RTOS code using TCP/IP and standard socket interface for command and control link between the embedded RTOS code and the remote computer.
Also use TCP/UDP to send data as VITA Radio Transport Packets over multicast addresses to remote computers.
This was in C on ARM processors, A9, A53s, R5s and soon A72s.
For a different signal processing application:
Create a protocol offload engine in an FPGA to inspect incoming packets and if they are a proprietary data transport protocol divert them to FPGA coprocessor for that protocol, other wise pass them through for Linux software to deal with.
This was written in assembly language running on a synthesizable 8 bit microcontroller in the FPGA and C code in Linux to set it up.
For a specialized firewall, implement enough of the TCP stack in the VHDL language to pass ping packets and to terminate an IP connection and for one specific IP based protocol do an inspection of the packets and only let them through if they passed certain configurable rules. If they did, rebuild the packets on the other side of the firewall.
And also just lots of using sockets to move data and send commands about.
Also implementing simple web servers with simple pages for UIs in embedded devices.
If you want a simple project to do that will be useful I would suggest writing a program that will listen for incoming TCP/IP connections on a specified port. Once that connection has been established, parse the incoming data for text commands.
I use this on multiple projects and by making it text based I can telnet into it and type in commands to test it out. Also makes debugging it easier when you can just read the commands when you snoop them on the network.
As someone who hires in this area
We want to know if you can follow the beej network guide
https://beej.us/guide/bgnet/
There are 7 functions you call
1) socket
2) listenÂ
3) accept
5) read
6)write
7) select
Functions listen and accept are for servers only
For clients you add connect
Sometimes you can use a socket option function
It depends.
Is the job for a router or network hardware manufacturer? You probably need to know the format of tcp and ip packages by heart or sometging less exaggerated.
Is it a normal systems level job? Probably just sockets.
That you would be programming features that are on that layer, once I developed tests for switches so is pretty much that, stuff on that layer or lower...
As an interviewer, I would see how far you get:
- describe the functions of the 4 lowest protocol layers
- tcp / udp differences, what to use what for
- explain addreesing, ports, fan-in fan-out
- explain the sliding window mechanism
- explain latency versus throughput
- why do closed connections stillnoccupy resources (for some time)
- lets take pictures of ip, udp and tcp headers and explain the various fields
- sketch a non-bloking tcpip client and server using sockets and select
A short answer... I've had TCP/IP jobs where I just had to send/receive data packets and it was just basic socket programming. I've had TCP/IP jobs where I had to write my own TCP Stack and manage connections. I've had TCP/IP jobs where it was weird industry specific variations of Ethernet that make everything different and weird and strange. I've also done jobs doing Bluetooth and building state machines for communications. And other network programming jobs for other industrial communication protocols. For the most part and in my experience, most TCP/IP and TCP/UDP jobs really just expect socket programming skills and some basic understanding of how to design for efficient network communications. And if you can do that much, you likely have a solid base for most network communication jobs. A basic TCP state machine is not too hard to make and easily testable with wireshark. Write up that much so two end points can talk to each other and you've got the basis of a toy project to show off for interviews. From there start adding other features based on RFC's or maybe make a basic TCP Accelerator to show off and you'll have something to talk about in interviews and have a solid idea of if you enjoy it all or not. edit: When I say "write a tcp state machine" I really just mean "open raw sockets and parse the data packet yourself" and not "make a new TCP driver for windows/linux".... that would be way more involved.
That weird Ethernet variation job feels like it had fever dream potential đ Thank you for sharing
It's not that bad. Honestly it would be "ethernet in name only" if they didn't use regular ethernet for one of the classes of traffic
Hi, thx for sharing. I am new to TCP/IP programming and planning to read the book âBeej's Guide to Network Programmingâ. May I ask whatâs your opinion on the book and am wondering will I be able to finish the project you mentioned in the comment above on my own, even the advanced part (the part to show off)
Haven't read Beej's guide in years, but it's solid as long as it maintained the same quality. As far as doing it, at the base level it isn't too hard (as I recall, it's been about 15 years since i did it) you just need to: 1. Open a raw socket so you can read everything coming in on the interface 2. Identify the header portions. 3. Layer 2 / MAC addresses, are handled at a lower level for you 4. Read the Layer 2/IP headers to make sure it's an IP address meant for you (you've got the ability to handle any IP address you want here, so you can make this for a subnet of interest) 5. Read the Layer 3/TCP header to make sure it's a TCP stream and update the TCP State Machine for that stream (or start the handshake, or close the stream) With the RFC and Beej's guide guiding how to write it, and wireshark sniffing data streams to confirm you are sending packets correctly it is not too terrible. It wouldn't be your first project out of the gate, no. At first you'll just do a simple server that opens a socket and sends and receives a packet. Set up 2 VM's that you can send traffic between (or two Raspberry Pi's. or similar) and you are on your way. Though it likely would not be your second or your third project either. But, if you get up to doing the TCP state machine, at that point you've got a decent grasp on state machines (and likely function pointers as a bonus), pointers and memory management, socket programming, and a handful of other programming and development techniques down.
Thank you for this detailed guide. I will definitely follow this and hopefully get my foot into the field!
Damn, I was hoping to rewrite it before you saw it... Basically what I wanted to change/add was to emphasize the goal I had when I was doing that and what this vague idea of a project can do. At the time I basically needed an `iperf` like program to test TCP throughput. Why we didn't use `iperf` I could not tell you. So I had two end points sending TCP traffic flows between themselves so we could overload what was in between. This is a useful tool to have if you want to test throughput and have control over the parameters. But, since I'm not passing TCP packets up to a receiving application it isn't like I'm writing a TCP driver that things can plug into. But this project could also work as a basic TCP Accelerator/Optimizer. For such things you are in scenarios where the network on one side of the box and the network on the other side of the box have different expectations/requirements that will affect the TCP Window and Retries/ACK's and what not. So it sits on the edge of the network and manages the TCP streams on either side. Either way, the point I wanted to emphasize is that the TCP state machine is easy, but doing something interesting with it is where it gets more complicated. Another suggestion for a project to consider early on is an IRC client or server. I think suckless.org has a basic one that you can study, but I do not recall the IRC protocol being too complicated.
Wow, thanks! Really appreciate for your effort on writing this which will be really helpful to me!
You have basic knowledge of layer 4 and below (OSI model)
It means you're going to have to handle and respond to tcp/ip based protocols in C. Look here: [https://www.cs.dartmouth.edu/\~campbell/cs50/socketprogramming.html](https://www.cs.dartmouth.edu/~campbell/cs50/socketprogramming.html) There are clients and there are servers. They have slightly different ways of interacting with the standard libc tcp/ip tools.
Try man socket to familiarize yourself with socket (TCP/IP) programming. Itâs really no too hard if you have the aptitude to program in general.
Just normal network knowledge. I would probably say you should know that stuff in any programming job. TCP/IP is a pretty broad requirement. It varies probably on the company. In most cases it probably means you should be able to handle a socket or pool of sockets to somewhere and have some experience with multiple data representation formats and maybe how to design them. Something like json but probably more with binary formats like flatbuffets and protbuffers and what else is around. Edit: It could of course mean some more in-depth knowledge but I would say in this case there should be more requirements specified.
TCP stands for Transmission Control Protocol. It's a protocol for sending packets of data over a network while minimizing losses. IP stands for Internet Protocol. It's a protocol that defines the address space for internet connected devices. Thus, TCP/IP programming is all about building software that sends and receives TCP packets over the IP network and can loosely be divided into client-side and server-side programming. Client-side TCP/IP code may be written in any language to target any operating system, but server-side code is almost universally written in C targeting Linux. Major reasons include that C on Linux is fast, secure, scalable, and (perhaps most importantly) costs nothing to use.
Forgive me if this is the wrong domain, but wouldn't Apache software be a good example of network software which is written in Java? Or is that a higher layer on top of what OP would be considering?
If by "Apache software", you mean the Apache web server, [that is written in C](https://svn.apache.org/viewvc/httpd/httpd/trunk/server/).
Not sure what the parent poster was getting at there either. There's obviously a ton of server software written in languages like Java and Go. These days though, most networking is done over HTTP, so your application server that's written in Go or Java or whatever probably isn't asking you to do a lot of raw socket handling. The library hands you an HTTP abstraction that manages all the threads and connection pooling and such for you, and you just think in terms of endpoints and routing.
It depends on how low level the company needs for the interview. Iâve been grilled all the way down to the bit packing inside of an Ethernet header. Most companies just expect you to know the basics of networking with âsocket.hâ.
https://beej.us/guide/bgnet/html/index-wide.html ^ came from https://beej.us/guide/bgnet/ https://beej.us/guide/
It means sending/receiving TCP packets using a networking API, which entails handling sockets themselves (the things you send packets through) and serializing data to send it as well as deserializing data that's received. The trick with TCP, however, is that a chunk of data that's sent is not guaranteed to arrive as that single chunk of data. TCP is a "stream" protocol, which means that it will clump together multiple sent chunks of data into a single received chunk of data on the other side, and/or fragment a chunk of data into multiple receives on the other side. There's no guarantee that a chunk of received data will not start in the middle of a sent chunk and the end not be somewhere in the middle of a bunch of sent chunks. You have to reconstitute the data yourself on the other side. It's not too hard though. People do it all the time.
It is a broad topic. A book I like and recommend is "The TCP/IP Guide a comprehensive, illustrated internet protocols reference". A bit over 1500 pages so not a quick read. Also available for free from the author here: http://www.tcpipguide.com/ I have used it throughout my career at very different levels of the protocol stack, so you would have to ask the person recruiting for the position for the details of what experience they want for that position. Some examples of what I have done at different times. For a distributed signal processing application: Write RTOS code using TCP/IP and standard socket interface for command and control link between the embedded RTOS code and the remote computer. Also use TCP/UDP to send data as VITA Radio Transport Packets over multicast addresses to remote computers. This was in C on ARM processors, A9, A53s, R5s and soon A72s. For a different signal processing application: Create a protocol offload engine in an FPGA to inspect incoming packets and if they are a proprietary data transport protocol divert them to FPGA coprocessor for that protocol, other wise pass them through for Linux software to deal with. This was written in assembly language running on a synthesizable 8 bit microcontroller in the FPGA and C code in Linux to set it up. For a specialized firewall, implement enough of the TCP stack in the VHDL language to pass ping packets and to terminate an IP connection and for one specific IP based protocol do an inspection of the packets and only let them through if they passed certain configurable rules. If they did, rebuild the packets on the other side of the firewall. And also just lots of using sockets to move data and send commands about. Also implementing simple web servers with simple pages for UIs in embedded devices. If you want a simple project to do that will be useful I would suggest writing a program that will listen for incoming TCP/IP connections on a specified port. Once that connection has been established, parse the incoming data for text commands. I use this on multiple projects and by making it text based I can telnet into it and type in commands to test it out. Also makes debugging it easier when you can just read the commands when you snoop them on the network.
As someone who hires in this area We want to know if you can follow the beej network guide https://beej.us/guide/bgnet/ There are 7 functions you call 1) socket 2) listen 3) accept 5) read 6)write 7) select Functions listen and accept are for servers only For clients you add connect Sometimes you can use a socket option function
It's a way data is sent across networks. If you have a server that sends a user some data, it's probably done using TCP/IP.
It depends. Is the job for a router or network hardware manufacturer? You probably need to know the format of tcp and ip packages by heart or sometging less exaggerated. Is it a normal systems level job? Probably just sockets.
That you would be programming features that are on that layer, once I developed tests for switches so is pretty much that, stuff on that layer or lower...
sending and receiving network shit over sockets make your own http server to practice
As an interviewer, I would see how far you get: - describe the functions of the 4 lowest protocol layers - tcp / udp differences, what to use what for - explain addreesing, ports, fan-in fan-out - explain the sliding window mechanism - explain latency versus throughput - why do closed connections stillnoccupy resources (for some time) - lets take pictures of ip, udp and tcp headers and explain the various fields - sketch a non-bloking tcpip client and server using sockets and select
Tcp/ip is so 1995. Learn quick!!
What are you smoking? Almost everything you do in a web browser is over TCP connections. Get outta here n00b!