trafpy.generator.src package

Subpackages

• trafpy.generator.src.dists package
  • Submodules
  • trafpy.generator.src.dists.node_dists module
  • trafpy.generator.src.dists.plot_dists module
  • trafpy.generator.src.dists.val_dists module
  • Module contents

Submodules

trafpy.generator.src module

trafpy.generator.src module

trafpy.generator.src module

trafpy.generator.src.builder module

trafpy.generator.src.demand module

trafpy.generator.src.flowcentric module

trafpy.generator.src.interactive module

trafpy.generator.src.jobcentric module

trafpy.generator.src.networks module

Module for generating and plotting networks.

trafpy.generator.src.networks.add_edge_capacity_attrs(network, edge, channel_names, channel_capacity, bidirectional_links=True)

Adds channels and corresponding max channel bytes to single edge in network.

Parameters

• network (networkx graph) – Network containing edges to whiich attrs will be added.

• edge (tuple) – Node-node edge pair.

• channel_names (list) – List of channel names to add to edge.

• channel_capacity (int,float) – Capacity to allocate to each channel.

• bidirectional_links (bool) – If True, each link has capacity split equally between src and dst port. I.e. all links have a src and dst port which are treated separately to incoming and outgoing traffic to and from given node (switch or server).

trafpy.generator.src.networks.add_edges_capacity_attrs(network, edges, channel_names, channel_capacity, bidirectional_links=True)

Adds channels & max channel capacitys to single edge in network.

To access e.g. the edge going from node 0 to node 1 (edge (0, 1)), you would index the network with network[0][1]

To access e.g. the channel_1 attribute of this particular (0, 1) edge, you would do network[0][1][‘channels’][‘channel_1’] OR if bidirectional_links, you do network[0][1][‘0_to_1_port’][‘channels’][‘channel_1’] or network[0][1][‘1_to_0_port’][‘channels’][‘channel_1] depending on which direction of the link you want to access.

Parameters

• network (networkx graph) – Network containing edges to which attrs will be added.

• edges (list) – List of node pairs in tuples.

• channel_names (list of str) – List of channel names to add to edge.

• channel_capacity (int, float) – Capacity to allocate to each channel.

• bidirectional_links (bool) – If True, each link has capacity split equally between src and dst port. I.e. all links have a src and dst port which are treated separately to incoming and outgoing traffic to and from given node (switch or server).

trafpy.generator.src.networks.gen_arbitrary_network(num_eps, ep_label=None, ep_capacity=12500, num_channels=1, racks_dict=None, topology_type=None)

Generates an arbitrary network with num_eps nodes labelled as ep_label.

Note that no edges are formed in this network; it is purely for ep name indexing purposes when using Demand class. This is useful where want to use the demand class but not necessarily with a carefully crafted networkx graph that accurately mimics the network you will use for the demands

Parameters

• num_eps (int) – Number of endpoints in network.

• ep_label (str,int,float) – Endpoint label (e.g. ‘server’). All endpoints will have ep_label appended to the start of their label (e.g. ‘server_0’, ‘server_1’, …).

• ep_capacity (int, float) – Byte capacity per end point channel.

• num_channels (int, float) – Number of channels on each link in network.

• racks_dict (dict) – Mapping of which end points are in which racks. Keys are rack ids, values are list of end points. If None, assume there is not clustering/rack system in the network where have different end points in different clusters/racks.

Returns

network object

Return type

networkx graph

trafpy.generator.src.networks.gen_channel_names(num_channels)

Generates channel names for channels on each link in network.

trafpy.generator.src.networks.gen_fat_tree(k=4, L=2, n=4, ep_label='server', rack_label='rack', edge_label='edge', aggregate_label='agg', core_label='core', num_channels=2, server_to_rack_channel_capacity=500, rack_to_edge_channel_capacity=1000, edge_to_agg_channel_capacity=1000, agg_to_core_channel_capacity=2000, rack_to_core_channel_capacity=2000, show_fig=False)

Generates a perfect fat tree (i.e. all layers have switches with same radix/number of ports).

Top layer is always core (spine) switch layer, bottom layer is always ToR (leaf) layer.

L must be either 2 (core, ToR) or 4 (core, agg, edge, ToR)

N.B. L=2 is commonly referred to as ‘2-layer Clos’ or ‘Clos’ or ‘spine-leaf’ topology

Resource for building (scroll down to summary table with equations):

https://packetpushers.net/demystifying-dcn-topologies-clos-fat-trees-part2/

Another good resource for data centre topologies etc. in general:

https://www.oreilly.com/library/view/bgp-in-the/9781491983416/ch01.html#:~:text=The%20most%20common%20routing%20protocol,single%20data%20center%2C%20as%20well.

Parameters of network:

• number of core (spine) switches = (k/2)^(L/2) (top layer)

• number of edge switches (if L=4) = (k^2)/2

• number of agg switches (if L=4) = (k^2)/2

• number of pods (if L=4) (pod is a group of agg and/or edge switches) = 2*(k/2)^(L-2)

• number of ToR (leaf) switches (racks) = 2*(k/2)^(L-1) (bottom layer)

• number of server-facing ToR ‘host’ ports = 2*(k/2)^2 (can have multiple servers connected to same host port, & can oversubscribe)

• number of servers = number ToR switches * n

Parameters

• k (int) – Number of ports (links) on each switch (both up and down).

• L (int) – Number of layers in the fat tree.

• n (int) – Number of server per rack.

• ep_label (str,int,float) – Endpoint label (e.g. ‘server’). All endpoints will have ep_label appended to the start of their label (e.g. ‘server_0’, ‘server_1’, …).

• edge_label (str,int) – Label to assign to edge switch nodes

• aggregate_label (str,int) – Label to assign to edge switch nodes

• core_label (str,int) – Label to assign to core switch nodes

• num_channels (int, float) – Number of channels on each link in network

• server_to_edge_channel_capacity (int,float) – Byte capacity per channel

• edge_to_agg_channel_capacity (int,float) – (if L==4) Byte capacity per channel

• agg_to_core_channel_capacity (int,float) – (if L==4) Byte capacity per channel

• rack_to_core_channel_capacity (int,float) – (if L==2) Byte capacity per channel

Returns

network object

Return type

networkx graph

trafpy.generator.src.networks.gen_nsfnet_network(ep_label='server', rack_label='rack', N=0, num_channels=2, server_to_rack_channel_capacity=1, rack_to_rack_channel_capacity=10, show_fig=False)

Generates the standard 14-node NSFNET topology (a U.S. core network).

Parameters

• ep_label (str,int,float) – Endpoint label (e.g. ‘server’). All endpoints will have ep_label appended to the start of their label (e.g. ‘server_0’, ‘server_1’, …).

• N (int) – Number of servers per rack. If 0, assume all nodes in nsfnet are endpoints

• num_channels (int,float) – Number of channels on each link in network.

• server_to_rack_channel_capacity (int,float) – Byte capacity per channel between servers and ToR switch.

• rack_to_rack_channel_capacity (int,float) – Byte capacity per channel between racks.

• show_fig (bool) – Whether or not to plot and show fig. If True, will display fig.

Returns

network object

Return type

networkx graph

trafpy.generator.src.networks.gen_simple_network(ep_label='server', num_channels=2, server_to_rack_channel_capacity=500, show_fig=False)

Generates very simple 5-node topology.

Parameters

• ep_label (str,int,float) – Endpoint label (e.g. ‘server’). All endpoints will have ep_label appended to the start of their label (e.g. ‘server_0’, ‘server_1’, …).

• num_channels (int,float) – Number of channels on each link in network.

• channel_capacity (int,float) – Byte capacity per channel.

• show_fig (bool) – Whether or not to plot and show fig. If True, will display fig.

Returns

network object

Return type

networkx graph

trafpy.generator.src.networks.get_endpoints(network, ep_label)

Gets list of endpoints of network.

Parameters

• network (networkx graph) – Networkx object.

• ep_label (str,int,float) – Endpoint label (e.g. ‘server’). All endpoints will have ep_label appended to the start of their label (e.g. ‘server_0’, ‘server_1’, …).

Returns

List of endpoints.

Return type

eps (list)

trafpy.generator.src.networks.get_fat_tree_positions(net, width_scale=500, height_scale=10)

Gets networkx positions of nodes in fat tree network for plotting.

trafpy.generator.src.networks.get_node_type_dict(network, node_types=[])

Gets dict where keys are node types, values are list of nodes for each node type in graph.

trafpy.generator.src.networks.init_global_network_attrs(network, max_nw_capacity, num_channels, ep_link_capacity, endpoint_label='server', topology_type='unknown', node_labels=['server'], racks_dict=None)

Initialises the standard global network attributes of a given network.

Parameters

• network (obj) – NetworkX object.

• max_nw_capacity (int/float) – Maximum rate at which info can be reliably transmitted over the network (sum of all link capacities).

• num_channels (int) – Number of channels on each link in network.

• topology_type (str) – Label of network topology (e.g. ‘fat_tree’).

• node_labels (list) – Label classes assigned to network nodes (e.g. [‘server’, ‘rack’, ‘edge’]).

• racks_dict (dict) – Which servers/endpoints are in which rack. If None, assume do not have rack system where have multiple servers in one rack.

trafpy.generator.src.networks.init_network_node_positions(net)

Initialises network node positions for plotting.

trafpy.generator.src.networks.plot_network(network, draw_node_labels=True, ep_label='server', network_node_size=2000, font_size=30, linewidths=1, fig_scale=2, path_to_save=None, show_fig=False)

Plots networkx graph.

Recognises special fat tree network and applies appropriate node positioning, labelling, colouring etc.

Parameters

• network (networkx graph) – Network object to be plotted.

• draw_node_labels (bool) – Whether or not to draw node labels on plot.

• ep_label (str,int,float) – Endpoint label (e.g. ‘server’). All endpoints will have ep_label appended to the start of their label (e.g. ‘server_0’, ‘server_1’, …).

• network_node_size (int,float) – Size of plotted nodes.

• font_size (int,float) – Size of of font of plotted labels etc.

• linewidths (int,float) – Width of edges in network.

• fig_scale (int,float) – Scaling factor to apply to plotted network.

• path_to_save (str) – Path to directory (with file name included) in which to save generated plot. E.g. path_to_save=’data/my_plot’

• show_fig (bool) – Whether or not to plot and show fig. If True, will return and display fig.

Returns

node distribution plotted as a 2d matrix.

Return type

matplotlib.figure.Figure

trafpy.generator.src.tools module

class trafpy.generator.src.tools.NumpyEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: JSONEncoder

Special json encoder for numpy types

default(obj)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

trafpy.generator.src.tools.calc_graph_diameter(graph)

Calculate diameter of a single graph.

trafpy.generator.src.tools.calc_graph_diameters(graphs, multiprocessing_type='none', print_times=False)

Calculate diameters of a list of graphs.

trafpy.generator.src.tools.compute_jensen_shannon_distance(p, q)

trafpy.generator.src.tools.gen_event_dict(demand_data, event_iter=None)

Use demand data dict to generate dict for each event in demand data.

trafpy.generator.src.tools.gen_event_times(interarrival_times, duration_times=None, path_to_save=None)

Use event interarrival times to generate event times.

trafpy.generator.src.tools.get_network_params(eps, all_combinations=False)

Returns basic params of network.

If all_combinations, will consider all possible pair combinations (i.e. src-dst and dst-src). If False, will consider src-dst==dst-src -> get half number of node pairs returned.

trafpy.generator.src.tools.load_data_from_json(path_to_load, print_times=True)

trafpy.generator.src.tools.pickle_data(path_to_save, data, overwrite=False, zip_data=True, print_times=True)

Save data as a pickle.

trafpy.generator.src.tools.save_data_as_csv(path_to_save, data, overwrite=False, print_times=True)

Saves data given as a csv file.

trafpy.generator.src.tools.save_data_as_json(path_to_save, data, overwrite=False, print_times=True)

trafpy.generator.src.tools.to_undirected_graph(directed_graph)

Converts directed graph to an undirected graph.

trafpy.generator.src.tools.unpickle_data(path_to_load, zip_data=True, print_times=True)

Re-load previously pickled data.

Module contents