Chord Module¶

class py2p.chord.chord_connection(*args, **kwargs)[source]¶

The class for chord connection abstraction. This inherits from py2p.mesh.mesh_connection

__init__(*args, **kwargs)[source]¶

Sets up a connection to another peer-to-peer socket

Parameters:	sock – The connected socket object server – A reference to your peer-to-peer socket outgoing – Whether this connection is outgoing (default: False)

id_10¶: Returns the nodes ID as an integer

collect_incoming_data(data)¶

Collects incoming data

Parameters:	data – The most recently received `bytes`
Returns:	`True` if the data collection was successful, `False` if the connection was closed

fileno()¶: Mirror for the fileno() method of the connection’s underlying socket

find_terminator()¶: Returns whether the defined return sequences is found

found_terminator()¶

This method is called when the expected amount of data is received

Returns:	`None`

handle_renegotiate(packets)¶

The handler for connection renegotiations

This is to deal with connection maintenance. For instance, it could be that a compression method fails to decode on the other end, and a node will need to renegotiate which methods it is using. Hence the name of the flag associated with it, “renegotiate”.

Parameters:	packets – A `tuple` containing the packets received in this message
Returns:	`True` if an action was taken, `None` if not

handle_waterfall(msg, packets)¶

This method determines whether this message has been previously received or not.

If it has been previously received, this method returns True.

If it is older than a preset limit, this method returns True.

Otherwise this method returns False, and forwards the message appropriately.

Parameters:	msg – The message in question packets – The message’s packets
Returns:	Either `True` or `False`

protocol¶: Returns server.protocol

send(msg_type, *args, **kargs)¶

Sends a message through its connection.

Parameters:	msg_type – Message type, corresponds to the header in a `InternalMessage` object args – A list of bytes-like objects, which correspond to the packets to send to you kargs – There are two available keywords: id – The ID this message should appear to be sent from (default: your ID) time* – The time this message should appear to be sent from (default: now in UTC)
Returns:	the `IntenalMessage` object you just sent, or `None` if the sending was unsuccessful

send_InternalMessage(msg)¶

Sends a preconstructed message

Parameters:	msg – The `IntenalMessage` you wish to send
Returns:	the `IntenalMessage` object you just sent, or `None` if the sending was unsuccessful

class py2p.chord.chord_daemon(*args, **kwargs)[source]¶

The class for chord daemon. This inherits from py2p.mesh.mesh_daemon

__init__(*args, **kwargs)[source]¶

Sets up a daemon process for your peer-to-peer socket

Parameters:	addr – The address you wish to bind to port – The port you wish to bind to server – A reference to the peer-to-peer socket
Raises:	`socket.error` – The address you wanted is already in use `ValueError` – If your peer-to-peer socket is set up with an unknown encryption method

handle_accept()[source]¶: Handle an incoming connection

kill_old_nodes(handler)¶: Cleans out connections which never finish a message

mainloop()¶: Daemon thread which handles all incoming data and connections

process_data(handler)¶: Collects incoming data from nodes

protocol¶: Returns server.protocol

class py2p.chord.chord_socket(addr, port, prot=<protocol object>, out_addr=None, debug_level=0)[source]¶

The class for chord socket abstraction. This inherits from py2p.mesh.mesh_socket

__init__(addr, port, prot=<protocol object>, out_addr=None, debug_level=0)[source]¶

Initializes a mesh socket

Parameters:

addr – The address you wish to bind to (ie: “192.168.1.1”)
port – The port you wish to bind to (ie: 44565)
prot – The protocol you wish to operate over, defined by a py2p.base.protocol object
out_addr – Your outward facing address. Only needed if you’re connecting over the internet. If you use ‘0.0.0.0’ for the addr argument, this will automatically be set to your LAN address.
debug_level – The verbosity you want this socket to use when printing event data

Raises:

socket.error – The address you wanted could not be bound, or is otherwise used

addr¶: An alternate binding for self.out_addr, in order to better handle self-references in pathfinding

data_storing¶

disconnect_least_efficient()[source]¶

Disconnects the node which provides the least value.

This is determined by finding the node which is the closest to its neighbors, using the modulus distance metric

Returns:	A `bool` that describes whether a node was disconnected

_handle_peers(msg, handler)[source]¶

This callback is used to deal with peer signals. Its primary jobs is to connect to the given peers, if this does not exceed py2p.chord.max_outgoing

Parameters:	msg – A `message` handler – A `chord_connection`
Returns:	Either `True` or `None`

dump_data(start, end)[source]¶

Parameters:	start – An `int` which indicates the start of the desired key range. `0` will get all data. end – An `int` which indicates the end of the desired key range. `None` will get all data.
Returns:	A nested `dict` containing your data from start to end

get(key, ifError=None, timeout=10)[source]¶

Looks up the value at a given key. Under the covers, this actually checks five different hash tables, and returns the most common value given.

Parameters:	key – The key that you wish to check. Must be a `str` or `bytes`-like object ifError – The value you wish to return on exception (default: `None`) timeout – The longest you would like to await a value (default: 10s)
Returns:	The value at said key, or the value at ifError if there’s an Exception

set(key, value)[source]¶

Updates the value at a given key. Under the covers, this actually uses five different hash tables, and updates the value in all of them.

Parameters:	key – The key that you wish to update. Must be a `str` or `bytes`-like object value – The value you wish to put at this key. Must be a `str` or `bytes`-like object

update(update_dict)[source]¶

Equivalent to dict.update()

This calls chord_socket.store() for each key/value pair in the given dictionary.

Parameters:	update_dict – A `dict`-like object to extract key/value pairs from. Key and value be a `str` or `bytes`-like object

find(key)[source]¶

Finds the node which is responsible for a certain value. This does not necessarily mean that they are supposed to store that value, just that they are along your path to said node.

Parameters:	key – The key that you wish to check. Must be a `int` or `long`

Returns: A chord_connection or this socket

find_prev(key)[source]¶

Finds the node which is farthest from a certain value. This is used to find a node’s “predecessor”; the node it is supposed to delegate to in the event of a disconnections.

Parameters:	key – The key that you wish to check. Must be a `int` or `long`

Returns: A chord_connection or this socket

next¶: The connection that is your nearest neighbor ahead on the hash table ring

prev¶: The connection that is your nearest neighbor behind on the hash table ring

_send_peers(handler)[source]¶

Shortcut method for sending a peerlist to a given handler

Parameters:	handler – A `chord_connection`

_send_meta(handler)[source]¶

Shortcut method for sending a chord-specific data to a given handler

Parameters:	handler – A `chord_connection`

join()[source]¶: Tells the node to start seeding the chord table

unjoin()[source]¶: Tells the node to stop seeding the chord table

connect(*args, **kwargs)[source]¶

This function connects you to a specific node in the overall network. Connecting to one node should connect you to the rest of the network, however if you connect to the wrong subnet, the handshake failure involved is silent. You can check this by looking at the truthiness of this objects routing table. Example:

>>> conn = mesh.mesh_socket('localhost', 4444)
>>> conn.connect('localhost', 5555)
>>> # do some other setup for your program
>>> if not conn.routing_table:
...     conn.connect('localhost', 6666)  # any fallback address

Parameters:	addr – A string address port – A positive, integral port id – A string-like object which represents the expected ID of this node

keys()[source]¶: Returns an iterator of the underlying dict‘s keys

__iter__()[source]¶: Returns an iterator of the underlying dict‘s keys

values()[source]¶

Returns:	an iterator of the underlying `dict`‘s values
Raises:	`KeyError` – If the key does not have a majority-recognized value `socket.timeout` – See KeyError

items()[source]¶

Returns:	an iterator of the underlying `dict`‘s items
Raises:	`KeyError` – If the key does not have a majority-recognized value `socket.timeout` – See KeyError

pop(key, *args)[source]¶

Returns a value, with the side effect of deleting that association

Parameters:	Key – The key you wish to look up. Must be a `str` or `bytes`-like object ifError – The value you wish to return on Exception (default: raise an Exception)
Returns:	The value of the supplied key, or `ifError`
Raises:	`KeyError` – If the key does not have a majority-recognized value `socket.timeout` – See KeyError

popitem()[source]¶

Returns an association, with the side effect of deleting that association

Returns:	An arbitrary association
Raises:	`KeyError` – If the key does not have a majority-recognized value `socket.timeout` – See KeyError

_chord_socket__connect(addr, port, id=None)¶

Private API method for connecting and handshaking

Parameters:	addr – the address you want to connect to/handshake port – the port you want to connect to/handshake

_chord_socket__handle_key(msg, handler)¶

This callback is used to deal with new key entries. Its primary job is:

Ensure keylist syncronization

Parameters:	msg – A `message` handler – A `chord_connection`
Returns:	Either `True` or `None`

_chord_socket__handle_meta(msg, handler)¶

This callback is used to deal with chord specific metadata. Its primary job is:

set connection state

Parameters:	msg – A `message` handler – A `chord_connection`
Returns:	Either `True` or `None`

_chord_socket__handle_request(msg, handler)¶

This callback is used to deal with request signals. Its two primary jobs are:

if you know the ID requested, respond to it
if you don’t, make a request with your peers

Parameters:	msg – A `message` handler – A `chord_connection`
Returns:	Either `True` or `None`

_chord_socket__handle_retrieve(msg, handler)¶

This callback is used to deal with data retrieval signals. Its two primary jobs are:

respond with data you possess
if you don’t possess it, make a request with your closest peer to that key

Parameters:	msg – A `message` handler – A `chord_connection`
Returns:	Either `True` or `None`

_chord_socket__handle_retrieved(msg, handler)¶

This callback is used to deal with response signals. Its two primary jobs are:

if it was your request, send the deferred message
if it was someone else’s request, relay the information

Parameters:	msg – A `message` handler – A `chord_connection`
Returns:	Either `True` or `None`

_chord_socket__handle_store(msg, handler)¶

This callback is used to deal with data storage signals. Its two primary jobs are:

store data in keys you’re responsible for
if you aren’t responsible, make a request with your closest peer to that key

Parameters:	msg – A `message` handler – A `chord_connection`
Returns:	Either `True` or `None`

_chord_socket__lookup(method, key, handler=None)¶

Looks up the value at a given hash function and key. This method deals with just one of the underlying hash tables.

Parameters:	method – The hash table that you wish to check. Must be a `str` or `bytes`-like object key – The key that you wish to check. Must be a `int` or `long`
Returns:	The value at said key, or an `py2p.utils.awaiting_value` object, which will eventually contain its result

_chord_socket__store(method, key, value)¶

Updates the value at a given key. This method deals with just one of the underlying hash tables.

Parameters:	method – The hash table that you wish to check. Must be a `str` or `bytes`-like object key – The key that you wish to check. Must be a `int` or `long` value – The value you wish to put at this key. Must be a `str` or `bytes`-like object

_get_peer_list()¶: This function is used to generate a list-formatted group of your peers. It goes in format [ ((addr, port), ID), ...]

_mesh_socket__clean_waterfalls()¶

This function cleans the deque of recently relayed messages based on the following heuristics:

Delete all duplicates
Delete all older than 60 seconds

_mesh_socket__handle_handshake(msg, handler)¶

This callback is used to deal with handshake signals. Its three primary jobs are:

reject connections seeking a different network
set connection state
deal with connection conflicts

Parameters:	msg – A `message` handler – A `mesh_connection`
Returns:	Either `True` or `None`

_mesh_socket__handle_request(msg, handler)¶

This callback is used to deal with request signals. Its three primary jobs are:

respond with a peers signal if packets[1] is '*'
if you know the ID requested, respond to it
if you don’t, make a request with your peers

Parameters:	msg – A `message` handler – A `mesh_connection`
Returns:	Either `True` or `None`

_mesh_socket__handle_response(msg, handler)¶

This callback is used to deal with response signals. Its two primary jobs are:

if it was your request, send the deferred message
if it was someone else’s request, relay the information

Parameters:	msg – A `message` handler – A `mesh_connection`
Returns:	Either `True` or `None`

_mesh_socket__resolve_connection_conflict(handler, h_id)¶

Sometimes in trying to recover a network a race condition is created. This function applies a heuristic to try and organize the fallout from that race condition. While it isn’t perfect, it seems to have increased connection recovery rate from ~20% to ~75%. This statistic is from memory on past tests. Much improvement can be made here, but this statistic can likely never be brought to 100%.

In the failure condition, the overall network is unaffected for large networks. In small networks this failure condition causes a fork, usually where an individual node is kicked out.

Parameters:	handler – The handler with whom you have a connection conflict h_id – The id of this handler

_send_handshake(handler)¶

Shortcut method for sending a handshake to a given handler

Parameters:	handler – A `mesh_connection`

_send_handshake_response(handler)¶: Shortcut method to send a handshake response. This method is extracted from __handle_handshake() in order to allow cleaner inheritence from py2p.sync.sync_socket

close()¶

If the socket is not closed, close the socket

Raises:	`RuntimeError` – The socket was already closed

copy()[source]¶: Returns a dict copy of this DHT

Warning

This is a very slow operation. It’s a far better idea to use items(), as this produces an iterator. That should even out lag times

disconnect(handler)¶

Closes a given connection, and removes it from your routing tables

Parameters:	handler – the connection you would like to close

handle_msg(msg, conn)¶

Decides how to handle various message types, allowing some to be handled automatically

Parameters:	msg – A `py2p.base.message` object conn – A `py2p.base.base_connection` object
Returns:	True if an action was taken, None if not.

incoming¶: IDs of incoming connections

outgoing¶: IDs of outgoing connections

recv(quantity=1)¶

This function has two behaviors depending on whether quantity is left as default.

If quantity is given, it will return a list of message objects up to length quantity.

If quantity is left alone, it will return either a single message object, or None

Parameters:	quantity – The maximum number of :py:class:`~py2p.base.message`s you would like to pull (default: 1)
Returns:	A list of `message` s, an empty list, a single `message` , or `None`

register_handler(method)¶

Register a handler for incoming method.

Parameters:	method – A function with two given arguments. Its signature should be of the form `handler(msg, handler)`, where msg is a `py2p.base.message` object, and handler is a `py2p.base.base_connection` object. It should return `True` if it performed an action, to reduce the number of handlers checked.
Raises:	`ValueError` – If the method signature doesn’t parse correctly

request_peers()¶: Requests your peers’ routing tables

send(*args, **kargs)¶

This sends a message to all of your peers. If you use default values it will send it to everyone on the network

Parameters:	args – A list of strings or bytes-like objects you want your peers to receive kargs – There are two keywords available: flag* – A string or bytes-like object which defines your flag. In other words, this defines packet 0. type – A string or bytes-like object which defines your message type. Changing this from default can have adverse effects.

Warning

If you change the type attribute from default values, bad things could happen. It MUST be a value from py2p.base.flags, and more specifically, it MUST be either broadcast or whisper. The only other valid flags are waterfall and renegotiate, but these are RESERVED and must NOT be used.

status¶

The status of the socket.

Returns:	`"Nominal"` if all is going well, or a list of unexpected (Exception, traceback) tuples if not

waterfall(msg)¶

This function handles message relays. Its return value is based on whether it took an action or not.

Parameters:	msg – The `message` in question
Returns:	`True` if the message was then forwarded. `False` if not.

py2p.chord.distance(a, b, limit=None)[source]¶: This is a clockwise ring distance function. It depends on a globally defined k, the key size. The largest possible node id is limit (or 2**384).

py2p.chord.get_hashes(key)[source]¶

Returns the (adjusted) hashes for a given key. This is in the order of:

SHA1 (shifted 224 bits left)
SHA224 (shifted 160 bits left)
SHA256 (shifted 128 bits left)
SHA384 (unadjusted)
SHA512 (unadjusted)

The adjustment is made to allow better load balancing between nodes, which assign responisbility for a value based on their SHA384-assigned ID.