In this tutorial, we’ll talk about INET, STREAM sockets. We’ll talk about what sockets are, and learn how to work with blocking and non-blocking sockets.
First things first, we’ll make a distinction between a “client” socket (endpoint of a conversation), and a “server” socket (switchboard operator).
Your client (e.g. your browser) uses only client sockets, and your server uses both client and server sockets.
1. Creating a Socket
To open a socket, you only need to import the module socket and do as follows:
client.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("www.webcodegeeks.com", 80))
When the connect is completed, s can be used to send in a request for the text of the page. It will read the reply and then be destroyed. Yes, destroyed. Client sockets are only used for one exchange (or a set of sequential exchanges).
In the server, a server socket is created to accept this connection:
server.py
import socket serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) serversocket.bind((socket.gethostname(), 80)) serversocket.listen(5)
Here, we are creating an INET, STREAM socket and binding it to our host at the 80 port. Notice the use of socket.gethostname(), as this is how you make your socket visible to the outer world. If you bind it to 'localhost' or '127.0.0.1', this will still be a server socket, but it only will be visible within the same machine. If you bind it to '', the socket will be reachable by any address the machine happens to have.
Also, low number ports are used by system’s services (HTTP, SMTP, etc.), if you want your socket and these services to work at the same time, you should stick to high number ports (4 digits)(8080, 9290, 9000, 8000, etc.).
Now, serversocket.listen, tells the socket how many connect requests should be queued before refusing outside connections. In other words, the argument to this method is the maximum active connections it will allow at the same time.
Using a while loop, we will start listening to connections:
server.py
import socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind((socket.gethostname(), 80))
serversocket.listen(5)
def something(clientsocket):
# Do something with the socket
return None
while True:
(clientsocket, address) = serversocket.accept()
something(clientsocket)
We defined a function that receives the socket, and then, within the while loop, we accept connections and send them to our function.
There are a couple of interesting things to notice there, though, like the fact that the server socket doesn’t actually send or receive data. It just produces client sockets which will communicate, through a dynamically assigned port which will be recycled after the exchange is done, with the actual client socket that connected to our server.
Another thing to have in mind here, is that something should be an asynchronous function. If the execution within the while loop blocks the main thread, you will see your response time next to a plane 9000 feet over the ground.
2. Using a Socket
Your client’s client socket and your web server’s client socket are identical, this is a peer to peer (P2P) conversation. The rules for this conversation are only defined by you as the designer. Normally the client starts this conversation by sending a request, but there are no rules for sockets.
There are two verbs to use for communication, send and recv, these operate on the network buffers. You hand bytes to them, but they won’t necessarily handle them all, as their major focus is the network buffer. They return when the associated network buffer has been filled (send) or emptied (recv), then they will tell you how many bytes they handled. It is your responsibility to call them again until your message has been completely dealt with.
When a recv returns 0 bytes, it means the other side has closed the connection. You will not receive any more data through this connection, although you may be able to send data successfully. But if you plan to reuse your socket for further transfers, you need to realize that sockets will not tell you that there is nothing more to read. Messages must either:
- Be fixed length: Not the best option.
- Be delimited: Even worse.
- Indicate how long they are: Now this is a solution.
- End by shutting down the connection: This might be a little violent.
As I said before, there are no rules with sockets, so you can choose any of these options to know when to stop reading from a socket. Although there are some ways that are better than others.
Discarding the fourth option (shutting down the connection), the simplest way of doing this is a fixed length message:
fixed_length_socket.py
import socket
class FixedLengthSocket:
def __init__(self, message_len, sock=None):
if sock is None:
self.sock = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)
else:
self.sock = sock
self.message_len = message_len
def connect(self, host, port):
self.sock.connect((host, port))
def send(self, msg):
total_sent = 0
while total_sent < self.message_len:
sent = self.sock.send(msg[total_sent:])
if sent == 0:
raise RuntimeError("socket connection broken")
total_sent += sent
def receive(self):
chunks = []
bytes_recd = 0
while bytes_recd < self.message_len:
chunk = self.sock.recv(min(self.message_len - bytes_recd, 2048))
if chunk == b'':
raise RuntimeError("socket connection broken")
chunks.append(chunk)
bytes_recd += len(chunk)
return b''.join(chunks)
3. Disconnecting
There is a method called shutdown that you are supposed to call before you actually close it. It is a message to the socket at the other end, which content depends on the flag you send as argument, where 1 is I won’t send anything else, but I’m still listening and 2 is Nope! I’m not listening anymore. Bye.
In Python, when a socket is garbage collected it will perform a shutdown if it’s needed, but you shouldn’t rely on this. A socket disappearing without performing a shutdown before, will cause the socket at the other end to hang indefinitely.
Now, if you are on the other side, you’ll see that the worst thing of working with sockets is when the other end just disappears without notifying you. Your socket will hang, as TCP is a reliable protocol and it will wait for a long time before giving up a connection.
In this case, if you are using threads, your thread is essentially dead, you can’t do anything about it. But, there is a bright side here: If you are not doing anything stupid, like holding a lock while doing a blocking read, the resources consumed by the thread are not something you should worry about.
Remember not to try and kill a thread, as they don’t automatically recycle resources. If you do manage to kill the thread, your whole process is likely to be screwed up.
4. A Little Example Here
To see these threads at work, we will create a sample program which receives incoming messages and echos them back to the sender.
server.py
import socket
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 10000)
print('starting up on %s port %s' % server_address)
sock.bind(server_address)
sock.listen(1)
while True:
print('waiting for a connection')
connection, client_address = sock.accept()
try:
print('connection from', client_address)
while True:
data = connection.recv(16)
print('received "%s"' % data)
if data:
print('sending data back to the client')
connection.sendall(data)
else:
print('no more data from', client_address)
break
finally:
connection.shutdown(2)
connection.close()
Here, we are creating a socket and binding it to localhost:10000 (this will only by available to programs running on the same machine), then we listen, at most, 1 connections at a time.
In the while loop, as the operation is fairly easy and fast, we are directly receiving and sending back the data with, as you can see, a fixed length.
Now, let’s write the client program:
client.py
import socket
import sys
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 10000)
print('connecting to %s port %s' % server_address)
sock.connect(server_address)
try:
message = b'This is the message. It will be repeated.'
print('sending "%s"' % message)
sock.sendall(message)
amount_received = 0
amount_expected = len(message)
while amount_received < amount_expected:
data = sock.recv(16)
amount_received += len(data)
print('received "%s"' % data)
finally:
print(sys.stderr, 'closing socket')
sock.shutdown(2)
sock.close()
Here we are creating a socket, but we don’t bind to anything, as this is a client socket. We connect to our server address and then start sending and receiving data.
You can see the output:
server.py output
starting up on localhost port 10000
waiting for a connection
connection from ('127.0.0.1', 52600)
received "b'This is the mess'"
sending data back to the client
received "b'age. It will be'"
sending data back to the client
received "b' repeated.'"
sending data back to the client
received "b''"
no more data from ('127.0.0.1', 52600)
waiting for a connection
client.py output
connecting to localhost port 10000 sending "b'This is the message. It will be repeated.'" received "b'This is the mess'" received "b'age. It will be'" received "b' repeated.'" <_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'> closing socket
There you can see the behaviour we talked about. They won’t process the entire message at once, they will process the data by chunks and you have to call as long as there is still data to process.
5. Non-blocking Sockets
If you reached this point, you now know most of what you need about sockets. With non-blocking sockets, you’ll use the same calls in the same ways.
In Python, to make a socket non-blocking you need to call socket.setblocking(0). You should do this after you create the socket, but before you use it (Notice the should).
The major difference between blocking and non-blocking sockets is that send, recv, connect and accept can return without having done anything. To work with this, se best solution is to use select.
ready_to_read, ready_to_write, in_error = select.select(
potential_readers,
potential_writers,
potential_errs,
timeout)
You pass select three lists: the first contains all sockets that you might want to try reading; the second all the sockets you might want to try writing to, and the last (normally left empty) those that you want to check for errors. You should note that a socket can go into more than one list. The select call is blocking, but you can give it a timeout. This is generally a sensible thing to do – give it a nice long timeout (say a minute) unless you have good reason to do otherwise.
In return, you will get three lists. They contain the sockets that are actually readable, writable and in error. Each of these lists is a subset (possibly empty) of the corresponding list you passed in.
If you have a “server” socket, put it in the potential_readers list. If it comes out in the readable list, your accept will (almost certainly) work. If you have created a new socket to connect to someone else, put it in the potential_writers list. If it shows up in the writable list, you have a decent chance that it has connected.
6. Download the Code Project
This was a basic example on sockets with Python.
You can download the full source code of this example here: python-sockets
