The most important configuration settings of Open Service Mesh (OSM)

Like most meshes, OSM leverages the ambassador pattern to work with your applications. This means that the mesh injects your application’s pod to create a sidecar proxy between your application container and the external world. Every call is therefore handled by the mesh’s sidecar instead of your application container directly. The sidecar pattern is widely used in K8s to handle cross-cutting concerns, not only to implement the ambassador pattern. The downside of sidecars is the overhead of memory and CPU consumption.

It appears that OSM’s configuration setting can dramatically impact the amount of memory used by the OSM sidecar. The default settings shown below can be overridden through the meshconfig custom resource definition.

spec: 

  certificate: 

    certKeyBitSize: 2048 

    serviceCertValidityDuration: 24h 

  featureFlags: 

    enableAsyncProxyServiceMapping: false 

    enableEgressPolicy: true 

    enableEnvoyActiveHealthChecks: false 

    enableIngressBackendPolicy: true 

    enableRetryPolicy: false 

    enableSnapshotCacheMode: false 

    enableWASMStats: true 

  observability: 

    enableDebugServer: true 

    osmLogLevel: info 

    tracing: 

      enable: false 

  sidecar: 

    configResyncInterval: 0s 

    enablePrivilegedInitContainer: false 

    localProxyMode: Localhost 

    logLevel: debug 

    resources: {} 

    tlsMaxProtocolVersion: TLSv1_3 

    tlsMinProtocolVersion: TLSv1_2 

  traffic: 

    enableEgress: true 

    enablePermissiveTrafficPolicyMode: true 

    inboundExternalAuthorization: 

      enable: false 

      failureModeAllow: false 

      statPrefix: inboundExtAuthz 

      timeout: 1s 

    inboundPortExclusionList: [] 

    networkInterfaceExclusionList: [] 

    outboundIPRangeExclusionList: [] 

    outboundIPRangeInclusionList: [] 

    outboundPortExclusionList: [] 
   

Two settings have a significant impact on memory consumption. These are enableWASMStats and enablePermissiveTrafficPolicyMode. It appears that setting them both to “True,” or, in other words, enabling them both, leads to excessive memory consumption and a chaotic situation where low-priority pods get killed by K8s.

To avoid this problem, the best and most secure option consists of disabling permissive mode and defining explicit authorizations when services must be able to communicate with each other.