Design and implement a TCP protocol semi-connected port scanner

A student fan sent a question:

From the title, you can tell that this student is majoring in network security.

Many fans think that Mr. Peng is driven by knowledge.

But as an old bird who owns many network protocol patents,

I am quite good at network knowledge!

Application layer sockets, networking, and network card drivers are all covered.

At present, there is still a lack of in-depth research on the Linux kernel protocol stack, which will be supplemented later.

1. Summary of the topic

The title requires scanning all TCP semi-connected ports. The functions that need to be implemented are as follows:

1. The attacker starts Task 1 and sends SYN packets to the specified server + port in a loop (the port number starts at 0 and increases in increments).
2. If there is a service on the server that has opened this port, it will reply with SYN+ACK , and the server will enter the SYN_RCVD state.
3. The attacker starts Task 2 and scans all received SYN+ACK packets. If the client receives SYN+ACK, it means that the server port is open. Task 2 can print out all open port information.

• Task 1: Using the socket API
• Task 2: Using the pcap library

2. TCP Basic Knowledge Points

To solve this problem, you must master the following knowledge points:

1. What is TCP
2. TCP 3-way handshake
3. What is a semi-join?
4. TCP, IP protocol header
5. How to use Libpcap library
6. Threads, processes

In general, the basic knowledge of network is still very high. This article will not list the basic knowledge points of TCP/IP protocol stack.

The following mainly strengthens the knowledge points of TCP involved in this topic.

1. TCP

First of all, we must understand the TCP protocol header:

• Sequence number: A random number generated by a computer when a connection is established as its initial value, which is transmitted to the receiving host through a SYN packet. Each time data is sent, the size of the "number of data bytes" is "accumulated". It is used to solve the problem of disordered network packets.
• Confirmation number: refers to the sequence number of the data that is "expected" to be received next time. After receiving this confirmation, the sender can assume that the data before this sequence number has been received normally. It is used to solve the problem of packet loss.
• Control bit: ACK: When this bit is 1, the "acknowledgement" field becomes valid. TCP stipulates that this bit must be set to 1 except for the SYN packet when the connection is initially established. RST: When this bit is 1, it indicates that an abnormality has occurred in the TCP connection and the connection must be forcibly disconnected. SYN: When this bit is 1, it indicates that a connection is desired to be established, and the initial value of the sequence number is set in its "sequence number" field. FIN: When this bit is 1, it indicates that no more data will be sent in the future and the connection is desired to be disconnected. When the communication ends and the connection is desired to be disconnected, the hosts on both sides of the communication can exchange TCP segments with the FIN position set to 1.

The most important field related to this topic is the control bit, and the operation of the control bit is mainly reflected in the 3-way handshake and 4-way handshake.

2. TCP three-way handshake

Initially, both the client and the server are in the CLOSED state, and then the server starts listening on a port and enters the LISTEN state:

• After the first handshake (SYN=1, seq=x), the client enters the SYN_SENT state.
• Second handshake (SYN=1, ACK=1, seq=y, ACKnum=x+1), after sending, the server enters SYN_RCVD state
• The third handshake (ACK=1, ACKnum=y+1), after sending, the client enters the ESTABLISHED state. When the server receives this packet, it also enters the ESTABLISHED state. TCP handshake, that is, data transmission can begin

3. TCP four times wave

Four wave processes:

• The client intends to close the connection. At this time, it will send a message with the FIN flag in the TCP header set to 1, that is, a FIN message. After that, the client enters the FIN_WAIT_1 state.
• After receiving the message, the server sends an ACK response message to the client, and then the server enters the CLOSED_WAIT state.
• After the client receives the ACK response message from the server, it enters the FIN_WAIT_2 state.
• After the server has processed the data, it also sends a FIN message to the client, and then the server enters the LAST_ACK state.
• After receiving the FIN message from the server, the client returns an ACK response message and then enters the TIME_WAIT state.
• After the server receives the ACK response message, it enters the CLOSE state, and the server has completed the connection closure.
• After 2MSL, the client automatically enters the CLOSE state, and the client also completes the connection closure.

4. TCP Status

The TCP protocol state transition diagram is as follows:

• CLOSED: indicates the initial state
• LISTEN: Indicates that a socket on the server is in listening state and can accept connections.
• SYN_RCVD: indicates that a SYN message has been received
• SYN_SENT: indicates that the client has sent a SYN message
• ESTABLISHED: Indicates that the connection has been established
• TIME_WAIT: Indicates that the FIN message from the other party has been received and the ACK message has been sent. It will return to the CLOSED available state after 2MSL.
• CLOSING: It means that after you sent a FIN message, you did not receive an ACK message from the other party, but received a FIN message from the other party. If both parties close a SOCKET at almost the same time, then both parties send FIN messages at the same time, that is, the CLOSING state will appear, indicating that both parties are closing the SOCKET connection
• CLOSE_WAIT: Waiting to close

5. Semi-connection/full connection

TCP semi-connected and fully connected states play an important role in server performance analysis, and usually reflect the processing capabilities of the server.

1) Semi-connection queue (syn queue)

The client sends a SYN packet, and the server replies with SYN+ACK after receiving it. The server then enters the SYN_RCVD state, and the socket is placed in the semi-connected queue.

2) Full connection queue (accept queue)

When the server receives the ACK from the client, the socket will be moved from the semi-connection queue to the full-connection queue. When the accpet function is called, the available socket will be returned to the user process from the head of the full-connection queue.

The full connection queue stores connections that have completed the TCP three-way handshake process and are waiting to be processed. The status on both the client and the server is ESTABLISHED

3. Packet capture example

If you want to learn network well, you must master the packet capture tool.

The following figure is a complete communication data packet of TCP 3-way handshake + HTTP GET request + 4-way handshake captured by Yikoujun through the packet capture tool.

 https://www.bilibili.com/video/BV1xr4y1T7cT/?vd_source=07570058 a62e0e8a6cf489efac35cfec​​​​​​​

Socket

For more information about the socket API, please refer to the following article: What is a socket?

5. libpcap

libpcap is a network data packet capture library with very powerful functions. The famous tcpdump under Linux is based on it.

Libpcap mainly consists of two parts: network tap and packet filter.

The network tap collects data from the network device driver for copying, and the filter decides whether to accept the packet.

libpcap uses the BSD packet filter (BPF) algorithm to filter the link layer data packets received by the network card.

The packet capture mechanism of libpcap is to add a bypass process at the data link layer. When a data packet arrives at the network interface, libpcap first obtains a copy of the data packet from the link layer driver using the created socket, and then sends the data packet to the BPF filter through the Tap function.

The BPF filter matches data packets one by one according to the filtering rules defined by the user. If a match is successful, the data packets are placed in the kernel buffer and passed to the user buffer. If a match fails, the data packets are discarded directly.

If no filtering rules are set, all packets will be put into the kernel buffer and passed to the user layer buffer.

1. libpcap installation

1. Online Installation

 sudo apt - get install libpcap - dev

This is suitable for friends who have internet access.

If you cannot install, try to update the source:

 sudo apt - get update 

2. Offline compilation and installation

 http://www.tcpdump.org/#latest-release​​​​​ 

 Then unzip
 tar zxvf libpcap - 1.10 .3 .tar .gz
 cd libpcap - 1.10.3
 ./configure
 sudo make
 sudo make install

2. Libpcap's packet capture process:

1. Search network devices: The purpose is to find available network cards. The function implemented is pcap_lookupdev(). If there are multiple network cards, the function will return a list of pointers to network device names.
2. Open the network device: Using the return value in the previous step, you can decide which network card to use, open the network card through the function pcap_open_live(), and return the number of seconds used to capture network data packets.
3. Get network parameters: Here we use the function pcap_lookupnet() to get the IP address and subnet mask of the specified network device.
4. Compile filtering strategy: The main function of Lipcap is to provide data packet filtering, which is implemented by the function pcap_compile().
5. Set the filter: Based on the previous step, use the pcap_setfilter() function to set it.
6. Use callback functions to capture data packets: functions pcap_loop() and pcap_dispatch() are used to capture data packets. You can also use functions pcap_next() and pcap_next_ex() to accomplish the same work.
7. Close the network device: pcap_close() function closes the device and releases resources.

3. Data structure description:

 struct pcap_pkthdr {
 struct timeval ts ; /* time stamp */
 bpf_u_int32 caplen ; /* Actual length of captured data packet*/
 bpf_u_int32 len ; /*packet length*/
 } ;

4. libcap library functions

A detailed explanation of libcap will be published later.

This article only talks about a few important functions.

• Open the network interface

 // This function will return the pcap_t type pointer of the specified interface, and all subsequent operations will use this pointer.
 pcap_t * pcap_open_live ( const char * device , int snaplen , int promisc , int to_ms , char * errbuf )
 device : Network interface string, which can be hard-coded directly , such as eth0.
 snaplen: For each data packet, how many bytes to capture from the beginning. We can set this value to capture only the header of each data packet without caring about the specific content. The typical Ethernet frame length is 1518 bytes, but some other protocols' data packets will be longer, but the length of a data packet of any protocol must be less than 65535 bytes.
 promisc: Specifies whether to enable promiscuous mode . 0 indicates non-promiscuous mode, and any other value indicates promiscuous mode. If promiscuous mode is to be enabled, the network card must also be enabled for promiscuous mode. You can use the following command to enable promiscuous mode for eth0: ifconfig eth0
 to_ms: The unit of capture time is milliseconds. 0 means waiting forever.
 errbuf : Output parameter, reason for failure to open the network interface. 

• Open offline pcap file

 pcap_t * pcap_open_offline ( const char * fname , char * errbuf )
 fname : File name.
 errbuf: error message when opening fails. 

• Packet capture function

 int pcap_loop ( pcap_t * p , int cnt , pcap_handler callback , u_char * user )
 p : Opened pcap_t type pointer.
 cnt: The total number of packets captured. If it is a negative number, the loop will continue.
 callback: callback function pointer
 user: The parameter passed to the callback function. 

 void callback ( u_char * userarg , const struct pcap_pkthdr * pkthdr , const u_char * packet )
 userarg: It is the last parameter of pcap_loop. When a sufficient number of packets are received, pcap_loop will call the callback function and pass the user parameter of pcap_loop ( ) to it.
 pkthdr: Captured message header information.
 packet: The data of the received packet. 

• Filter function compilation

 int pcap_compile ( pcap_t * p , struct bpf_program * fp , char * str , int optimize , bpf_u_int32 netmask )
 // fp: This is an outgoing parameter, storing the compiled bpf
 // str: filter expression
 // optimize: whether to optimize the filter expression
 // metmask: Simply set it to 0 

• Set the filter function

 int pcap_setfilter ( pcap_t * p , struct bpf_program * fp )
 // Parameter fp is the second parameter of pcap_compile ( ) , which stores the compiled bpf 

• Release the network interface

 void pcap_close ( pcap_t * p )
 // This function is used to close the pcap_t network interface object obtained by pcap_open_live ( ) and release related resources. 

• Open network package save file

 pcap_dumper_t * pcap_dump_open ( pcap_t * p , const char * fname )
 // p : is the network device we have opened and receives data packets from this device.
 // fname : is the file name we want to write, just name it randomly.
 // return : If an error occurs, NULL will be returned. This can be used to check whether the file is open. 

• Write network packets to a file

 void pcap_dump ( u_char * user , const struct pcap_pkthdr * h , const u_char * sp )
 user : It is the file descriptor dumpfp, but some type conversion needs to be done.
 Since this function is generally used in the packet_handler pointed to by the function pointer of pcap_loop ( ) , the user in packet_handler is the user here.
 h : pkt_header 

• Network packet file closed

 pcap_dump_close ( pcap_dumper_t * t ) ;

5. libcap filtering rules

Some examples of filter expressions are as follows:

• Only receive packets with source IP address 192.168.1.177

 src host 192.168.1.177

• Only receive TCP/UDP packets with destination port 80

 dst port 80

• Only receive packets that do not use the TCP protocol

 not tcp

• Only accept packets with the SYN flag set and a destination port of 22 or 23 (the 13th byte from the beginning of the TCP header)

 tcp [ 13 ] == 0x02 and ( dst port 22 or dst port 23 )

• Only receive ICMP ping request and ping response packets

 icmp [ icmptype ] == icmp - echoreply or icmp [ icmptype ] == icmp - echo

• Only receive packets with Ethernet MAC address 00:e0:09:c1:0e:82

 ether dst 00 : e0 : 09 : c1 : 0 e : 82

Only receive packets with IP ttl=5 (the 8th byte from the beginning of the IP header)

 ip [ 8 ] == 5

This example only captures data packets whose IP address is the local IP, and then the program parses the data packet protocol header:

 host 192.168.0.113​

6. Design plan

Implementation principle:

The atach and cap processes run in Ubuntu. The target terminal to be attacked can be any device in the network, as long as it can be pinged. This example is tested on Windows, and the bridge mode is used to bridge the Ubuntu network port and the Windows network port.

The main functions of the atach process are:

• Creating a TCP Socket
• Set the ip+port of the terminal to be attacked, and then execute the connect function
• If the connect is successful, it means that the port can be used by the other party.
• Modify the port value and repeat the previous 3 steps

The main functions of the cap process are:

• Through eth0, capture the specified rule: host 192.168.0.116 data packet
• Parse the Ethernet header, TCP header, IP header, and TCP header, and identify all packets with the sync+ack bit set to 1 in the TCP header
• Print out the data packets filtered out in step 2

Code flow:

7. Testing

1. Environment:

 Windows IP : 192.168.0.116
 ubuntu ip : 192.168.0.113

2. Documents:

 peng@ubuntu : ~/ work / test / pcap$ ls
 atach header .c libpcap - 1.10 .3 .tar .gz cap .c
 cap libpcap - 1.10 .3 atach .c protocol .h

Among them, atach is the attacker, used to send sync packets to the specified IP address. cap is used to detect the sync+ack packets received by all network cards. The program runs in Ubuntu.

3. Start the network debugging assistant

Start the Network Debugging Assistant on Windows.

Create several TCP Servers with port numbers 55, 56, and 57

insert image description here

4. Start the program

1) First start cap

 peng@ubuntu : ~/ work / test / pcap$ sudo ./cap 192.168 .0 .116
 Found device : eth0
 netaddr : 0000a8c0
 try to open device eth0
 filter : host 192.168.0.116

2) Start the attack program atach

You need to open a new terminal.

 peng@ubuntu : ~/ work / test / pcap$ ./atach 192.168 .0 .116

5. The running screenshots are as follows:

As can be seen in the log on the right, all accessible ports are listed, including 55, 56, and 57.

Note: The word atach is intentionally missing a t, otherwise it will not compile:

You can try your compiler and see if it works!

8. Code

The code has been synchronized to gitee, the address is as follows:

 https://gitee.com/yikoulinux/pcap.git​​​​​