Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Document status
DRAFT
Document owner
Owner Details

+972-54-2891166,   ab600d@intl.att.com


Table of Contents
absoluteUrltrue

...

Keep Alive

  • Each Configurable time (100 500 Milliseconds currently) E2M sends broadcast KA Request to all living E2T Instances through the RMR.

  • Every E2T Instance which gets it, responds with KA Response.

  • If any E2T Instance doesn’t respond for a configurable time (400 1500 Milliseconds currently) - E2M “kills” it:

    • E2M asks the K8S to kill the E2T Instance.

    • E2M disassociate the RANs associated with this “dying” instance, signs them as disconnected and Updates the RM about the disassociation of the RANs and about the deletion of the dead E2T Instance.

  • There is some logic how to sync between parallel flows - Killing an E2T Instance parallel to E2 Initialize arrives. See below.

...

E2M manages States that E2T Instance can be in the DB:

  • Active (normal state). Selector mechanism can choose it.

  • To be deleted - KA moves it to this state in DB the moment is understand it is dead. Selector mechanism can’t choose it. It is stay in stayed in this state until Routing manager success (in this case the entire entry is deleted) or failure, which is moved to the next stepE2T shutdown process completed. 

In case E2T initialize arrives and this E2T is in “To be Deleted” state - Don’t do anything. E2T will send another E2T initialize in case no KA during 300 milliseconds after E2 Initialize.

...

current State / trigger

State = Active

State =To be Deleted

KA decide to kill E2T instance

  • Move to “To be deleted” State;
  • Try to find POD in K8S and if found kill it; Even if fails - continue.
  • Disassociate all RANs, Delete the E2T Address at all.
  • Update RM, if fail – error but continue
  • In order to distinguish between 2 sequences KA cycle, KA updates the Timestamp when it moves the RAN to  this state, and sees it wasn't stayed in this state too long. It is called E2 Instance Deletion Timeout MS (Micro seconds) 

(Transient State, should not occurs)

  • Do all the steps Just like in Active (Previous Cell), except updating the state which is already “To be Deleted”.

E2 Initialize

  • Don’t update RM

  • Disassociate all RANs and inform RM

  • Ignore: E2T will send another E2 Initialize if no KA Request after previous E2 Initialize for some times.

KA Response

  • Update Timestamp

  • Ignore. This Instance is about to deleted

E2M Restart

  • Update Timestamp
  • ?????

Association RAN to Instance

  • There are 2 places where E2M search for a living E2T Instance to associate it with RAN:

    1. When a Setup Request from Dashboard arrives. This is for New RAN or for existing RAN which isn’t associated with any E2T Instance

    2. When KA kills an E2T Instance and divides its RANs to other living E2T Instances

  • E2M chooses the E2T Instance with the least # of RANs.

  • Anyhow, until the Routing Manager isn’t synced with this association, it isn’t stored in E2M DB

  • Setup Request from Dashboard which fails either in choosing any E2T Instance or updating the Routing Manager, either for New RAN or existing which isn’t associate with any E2T Instance , returns special rejection error is telling the Dashboard to try later.

  • In case KA failure in choosing the E2T to the RAN, the RANs are inserted to Orphan List. See below how E2M handle with Orphan List. It is usually when the last E2T Instance fails.

Dissociation RAN

  • The only place where RAN is associated with E2T address is when Setup Request arrives.

  • It is stored in DB -  in 2 places

  1. This RAN is added to the list of RANs served in the E2T Object
  2. The E2T Address is added to the RAN Object
  • The RM is informed on this Association between RAN and the E2T address
  • But also if RM isn't accessible, E2M doesn't rollback nor sending Setup Failure to remote RAN - since the simple way to ask the E2T to send Setup Failure not through the normal MEID flow is quite complicated.

Dissociation RAN

  • These are the

  • RAN which is already associated with E2T Instance, stays with this association, even if it lost connection. E2M will try to re-new the Setup until Max Retries (Configurable) reached.

  • There are 2 places where E2M dissociate RAN:

    1. When the Max Retries of sending Setup arrives, as a result of Lost Connection. The RAN is disassociate.When a request to Red Button arrives (See below)Lost Connection arrives from E2T

    2. When existing E2T Instance restart and E2M receives E2T Initialize message

    3. When KA decides to kill an E2T Instance
    4. When Red Button tears down all RANs.
  • In all these places - E2T address is cleared from the RAN(s) object(s), the RAN(s) removed from the list of RANs served in the E2T Object
  • The RM is informed

Interface to Routing Manager

  • Routing Manager RM updates the Routing Table in the following ways:

    1. Add or Delete E2T Instance - There are 2 RMR messages that are “Broadcast”:

      1. KA Request

      2. Clear all Connection (“Red Button”)

    2. Associate RAN(s) to E2T Instance (it can overrides previous association)

    3. Delete RAN association

  • E2M updates the Routing Manager RM in the following flows:

    1. When a new E2T Instance Initialized.

    2. When a a new RAN is added Setup Request arrived from the RAN and associated to existing E2T Instance

    3. When KA kills existing E2T Instance - in this update it also associates list of RANs to other existing E2T Instance (Or dissociate disassociates list of RANs since it was the last E2T Instance) and also delete E2T Instance

    4. When RAN lost connection and it exceed the Max Retry.connection  

    5. When a Red Button is requested - E2M dissociate all RANs

    6. When existing E2T Instance initialized, it is the first, and list of Orphans “are waiting” for it - E2M associates list of RANs to this E2T Instance it disassociates list of RANs served by this Instance

  • Also, when RM initialized, it queries E2M on the mapping between E2T and RANs and update the RIC with the new RMR.  See E2 Manager R3 Interface Design#GetAllE2TRequest
  • There are 4 REST APIs Routing Manager published. U can read their Swagger at Routing Manager Swagger

...

  • When Red Button request from Dashboard arrives, E2M inform Routing Manager to disassociate all RANs

  • E2M sends “broadcast” the Close all connections to all E2T Instances.

Orphans

  • As described above, when last E2T Instance fails, its RAN are entering the Orphan List.

  • The KA Job, after sending the KA request and after checking there is no any E2T which doesn’t respond and need to be killed. examines the Orphan List is not empty and there is a E2T Instance to associate them. If yes, it is updating the Routing Manager and if OK - sends Setup on each RAN

  • Also, on Red Button, we empty the Orphan List.

RAN State, Association and Orphan

To summarize, RAN can be in the following:

Associated with E2 Instance

RAN can be in the following state:

  • Connected

  • Connected Failure

  • Disconnect, but with Count of Retry less then Max, Usually this is in case RMR Failure

Associated with the Orphan List

RAN should be in a Disconnect State, while Count of Retry is 0.

Disassociated

RAN can be in the following state:

  • Shut Down

  • Disconnect, but with Count of Retry equal to Max