Monday 12 October 2020

New Local CDN, 13 Gbps of Traffic, 100.000% Annual Uptime, EFRIS Concerns, and Other Updates

New Peer: Al Bayan Media
The UIXP would like to welcome Al Bayan Media to the exchange! Al Bayan is a locally owned IPTV provider. They are peering publicly via our route servers in order to deliver a high quality service to their customers across all networks in Uganda. If your network is also peering with our route servers, it should already be exchanging traffic with Al Bayan.

If you would like to establish a bilateral session or market Al Bayan’s IPTV service to your customers in order to promote utilization of local traffic, please let us know and we will arrange an introduction.


Peak Traffic Level: 13 Gbps


The UIXP’s traffic has returned to growth following a brief reduction caused by the strict COVID-19 lockdown measures imposed earlier this year. Our daily peak traffic level now exceeds 13 Gbps during some weekdays as can be seen in the graph above.


Annual Uptime: 100.000%
The UIXP recently achieved an annual up-time of 100.000%. This is an improvement over our previous record of 99.994% which was achieved in March. Reliability is a critical growth enabler for the UIXP so we aim to maintain a similarly high level of availability going forward.

This milestone is the result of equipment upgrades that were made possible by our port-fee based sustainability model and we are grateful to everyone who supported its implementation.


IPv6 Peering Reminder
We would like to remind everyone that the UIXP supports IPv6 peering. At the moment, only four out of the twenty six networks that peer with our route servers use IPv6 and we would like to increase this number. We will reach out to some of you directly in the coming months in order to encourage adoption, but if you would like to proactively enable this for your network, please let us know and we will gladly assist.


Conrad Ekisa Moves to Ireland

Conrad Ekisa

Conrad Ekisa, one of our esteemed engineers, has left Uganda for some post-graduate research in Ireland. He’s still part of our team but will transition to a more limited remote support role in order to focus on his studies.

Contrad has made very significant contributions to the UIXP that greatly improved our quality of service, customer experience, and internal organisation. We are sad to see him go but we know that he’s destined for great things and wish him well on the journey ahead.


Internal System Upgrades

We recently upgraded our hypervisor cluster to the latest version of its software in order to take advantage of new features, stability improvements, and security updates. We have also made a few other changes to our network and systems that should significantly improve the overall security of our infrastructure.

Looking forward, we are exploring a potential deployment of an open-source ERP system in order to automate more of our invoicing and accounting processes. We also aim to upgrade our core exchange management system in order to take advantage of new features that are available in the latest release. We’ll share more information about this as we progress.


Raxio Expansion: Latest News
Earlier this year we signed a deal to extend our peering network into Raxio’s carrier neutral data centre in Namanve. This will allow us to attract new peers, provide redundancy for the networks that rely on us, and facilitate large-scale content and cloud deployments in Uganda.

As a reminder, we plan to deploy a fully independent IXP node at Raxio and interconnect it with our existing node at Communications House. This will allow us to offer a unified peering LAN that spans both locations. Network operators will be free to peer at either, or both, as needed.

After consulting with other IXPs, we have decided to prioritise Arista’s switching platform for the new site due to its deep buffers and support for advanced automation through an integration with our existing exchange management system. ISOC has generously agreed to sponsor this procurement and we are working to negotiate an order through Arista’s authorized local supplier.

We have also reached a tentative agreement for a dark fibre link between the two sites. This will require us to pay a monthly fee, but we have negotiated a discounted rate and are confident that we can afford the cost so long as collections remain consistent. We plan to finalize and execute the contract for this once we get closer to deployment.


URA: Concerns About EFRIS
We continue to remain in good standing with URA despite the issues reported in our previous newsletter. Following a brief dialogue with their officers, they informally stopped imposing penalties on us for using a cash based reporting method. We do not know how long this reprieve will last, but we’re grateful for URA’s apparent accommodation.

However, we have serious concerns related to the upcoming implementation of EFRIS, a centralized online accounting system which all companies operating in Uganda will be required to use for invoicing and payments. In order to use the system, every company in Uganda will be required to import and maintain its list of materials, products, and services along with all associated pricing information. To learn more about EFRIS, see here: http://efris.ura.go.ug/

In order to comply, the UIXP will either be forced to (a) duplicate all invoicing and accounting work or (b) migrate to an alternative accounting system which can integrate with EFRIS via an API which would undoubtedly come at a cost.

We are also concerned that EFRIS will create a single point of failure for the national economy, dramatically increase the burden of tax compliance, and potentially expose sensitive information to the database operators and others who might inappropriately gain access to the system.

The deadline for implementation was July 1, 2020 but has been extended to January 1st, 2021 following complaints from the Uganda Manufacturers Association (UMA) and others.


Payments & Collections
Collections have generally been good although we did experience some delays during Q2 and Q3 as COVID significantly disrupted the income and working conditions for a number of network operators in our market. Once again, we thank everyone for the continued support!


Community Engagement
We remain committed to community development despite the limitations imposed by COVID. Members of our team have both organised and spoken at a number of virtual events this year including AfPIF’s Virtual Peering Series. We will also be speaking at the upcoming Uganda Network Operators Group (UGNOG) launch event. In addition, we recently made a donation to the ISP Association of Uganda (ISPAU) in order to support their industry development efforts.

That’s all for now. If you have any questions, please contact us.

Tuesday 7 April 2020

New Uptime Record; New Traffic Record; and Raxio Expansion Plans

It's Q2 2020 and we have a lot of good news to share: We achieved a new uptime record, observed new peak traffic levels, and are expanding into Raxio's new carrier neutral data centre. Read on for details:

Coronavirus (COVID-19)
We would like to assure everyone that we are fully prepared to maintain operations during the lockdown period regardless of its duration. We have always had the ability to conduct most of our work remotely and the government has authorized a travel pass for us in order to accommodate any emergencies that require us to physically interact with our equipment.

Expansion Plans

We are excited to announce that we have signed a deal to extend our peering network into Raxio’s carrier neutral data center in Namanve. This will allow us to attract new peers, provide redundancy for the networks that rely on us, and make large-scale content and cloud deployments viable in Uganda.

To accomplish this, we will deploy a fully independent IXP at Raxio and interconnect it with our existing IXP at Communications House. This will allow us to offer a unified peering LAN that spans the two locations. Network operators will be free to peer at either, or both, as needed.

We believe this deal represents a significant milestone in the development of Uganda’s Internet ecosystem. Internet exchange points have a symbiotic relationship with carrier neutral data centers and we expect that this deal will catalyze a virtuous cycle of growth that will make the Internet significantly cheaper, faster, and more reliable for all.
 


Please visit Raxio's website for a copy of the press release.

New Stability Record: 99.994%
We’re extremely happy to report that we have achieved 99.99% uptime since taking direct control of our power and air conditioning.
This success would not have been possible to achieve without everyone’s support for our sustainability model -- so thanks to all once again!
 
Here are some graphs to demonstrate the UIXP’s reliability before and after:


BEFORE: 2018-06-01 to 2019-06-01
97.891% uptime (7d 17h 17m 46s of downtime)


AFTER: 2019-06-01 to 2020-04-02
99.994% uptime (26m 7s of downtime)

Note that the “BEFORE” data represents an extremely conservative downtime estimate as it does not account for partial outages including a multi-week CDN outage caused by an air conditioning system failure.

New Peak Traffic Record: 11Gbps
We recently observed the highest peak traffic levels ever recorded in the UIXP’s history:
 

However, the graph also indicates a significant reduction in peak traffic levels following the outbreak of Coronavirus (COVID-19) due to the Week 13 lockdown. We believe this occurred for the following reasons:

  • The lockdown closed businesses and schools which caused a mass migration of users away from uncapped fixed connections to pay-by-the-byte mobile connections while simultaneously evaporating disposable income.
     
  • Mobile Internet usage has less impact on the IXP because mobile carriers have on-net CDN cache nodes.

  • Many companies were not able to implement work-from-home solutions, and many popular video conferencing solutions route traffic via relay servers hosted in other countries.

Tax Challenges
While we are generally in good standing with the Uganda Revenue Authority (URA), we have encountered an apparently insurmountable problem regarding the timing of our VAT payments that we thought was worth sharing.

URA expects companies to pay VAT when invoices are issued regardless of when payments are received. In other words, if we issue $100,000 worth of invoices in January, URA expects us to pay $18,000 before we receive any payments.

We cannot accommodate this for at least two reasons:
  1. We cannot pay URA with money we don’t have. We are in a similar situation to most new businesses in that we don’t have enough free cash available to make advance tax payments.
  2. We can’t force networks to pay because the UCC has prohibited us from doing so. This means that we cannot guarantee that we will be able to recover any advance tax payments even if we had enough free cash to make them.
In light of the above, we are unable to meet URA’s expectations and have opted to pay VAT based on when we receive payments instead. This unfortunately means that we must occasionally pay fines and perform additional administrative work when network operators claim VAT against invoices based on the date of the invoice.

We have attempted to lobby URA about this issue without success. If anyone has any suggestions for how to resolve this problem, please let us know. In the mean-time, we encourage network operators to file any VAT claims against our invoices using the date the payments are made instead of the invoice date.
 
Collections
We are happy to report that all but three non-pro-bono networks have agreed to participate in our sustainability model. We are actively working to find ways to encourage these networks to come on-board, but they are generally either unresponsive or actively refuse to pay. Our options are unfortunately somewhat limited as the UCC ordered us to provide free service to these networks beyond what is available in our free service tier even though we have made it clear that our paying members are unhappy subsidizing their service.

Cash Reserves
We successfully established cash reserves of $75,000 and have filed a board resolution that formalizes this. Any draw-down now requires majority approval from the board and a plan to replenish the reserves. It also requires the board to implement an emergency budget if the reserves fall below 75% of the target level for one year.

Regulatory Environment
We do not have any news to report regarding the UCC's proposed IXP licensing framework and related telecommunications regulations. There has been little movement on this issue, likely due to senior staffing changes within government agencies and the COVID-19 situation. However, we continue to seek an amicable way to resolve the issue and will provide updates if the situation evolves.

Governance Reform

We still have not received any feedback on the draft constitution proposed to the community in 2015 and again, in updated form, in 2018.


If you are interested in governance reform, please review the draft and submit feedback (or forward it to your legal teams for the same) so that we can move the process forward.
A copy of the draft proposal and a summary document can be found here: https://uixp.co.ug/documentation/governance-reform-updated-draft-memarts-2018-12
 

Note that we have hired an additional technology oriented legal resource and plan to work with them on future revisions of the draft constitution.

That’s all for now. If anyone has any questions, please don’t hesitate to contact us.