In the previous post, we went over creating a functioning menu that allows you to swap between panels at will.

(Spoilers for these blog posts: This is the core of a library I wrote, Menutee, available here.)

The Halting Problem¹

That’s all well and good, but once you start integrating with other plugins or have other functionality, it doesn’t fit in the neat little paradigm we’ve set up. I ran into this issue when trying to make a note interface for one of my games. I wanted to pause the game when the player was reading a note. It’s simple, you set the time scale to 0, and there’s no problem.

What happens if the player wants to bring up our pause menu over that note?

1. The player brings up the note interface, pausing the game and unlocking the mouse cursor.
2. The player brings up the pause menu, caching the current time scale and mouse cursor (no change).
3. The player closes the pause menu, restoring the paused time scale and unlocked cursor (no change).
4. The player finally closes the note interface, unpausing the game and locking the mouse cursor.

This works using our existing menu, but every menu that we generate will need to cache the time scale, pause reference value, and cursor mode before applying it’s preference, and it will need to restore it to its previous state once it goes away. This is boilerplate that I’d prefer not to have across all my classes.

Let’s also look at an example of a menu from a plugin. The rewired input plugin has a control mapping menu that you can drop into your project. It’s great and it works (although it too requires some customization to look acceptable), but it controls the menu by itself. It operates through calls to start the menu and stop the menu, with very little customization of how the menu appears.

Is there a way that we can solve both of these problems at once?²

The State

From our example above, it’s clear that we want menus to be able to alter four components of our game state: the time scale, whether or not it’s paused,³ the cursor visibility, and the cursor lock mode.

So, let’s make a struct that encapsulates all those options. We’ll call it MenuAttributes.

public struct MenuAttributes {
    public CursorLockMode cursorLockMode;
    public bool cursorVisible;
    /// <summary>
    /// Sets the time scale while in the menu. If the value is negative, it
    /// will not modify the existing time scale.
    /// </summary>
    public float timeScale;
    public bool pauseGame;

    public static MenuAttributes StandardPauseMenu() {
        MenuAttributes attributes = new MenuAttributes();
        attributes.cursorLockMode = CursorLockMode.Confined;
        attributes.cursorVisible = true;
        attributes.timeScale = 0f;
        attributes.pauseGame = true;
        return attributes;
    }
}

MenuAttributes is an object that a menu will hold to tell the menu stack (coming later) what mode it wants to operate in.

The Interface

public interface IMenu {
    MenuAttributes GetMenuAttributes();
    void SetMenuUp(bool newUp);
    void SetMenuOnTop(bool isTop);
}

IMenu is an interface that informs the menu stack the attributes it wants to support, but also provides callbacks to the menu object so that it can reveal or hide itself depending on whether the menu is up or not.

The Arbiter

MenuStack is how all of the menus will be controlled. You drop it in the scene and reference it statically (similar to how Unity’s EventSystem works). Let’s take a look at the structure of the class.

public class MenuStack : MonoBehaviour {
    public static MenuStack Shared;

    [Header("Default Settings")]
    public CursorLockMode DefaultLockMode = CursorLockMode.Locked;
    public bool CursorVisible = false;

These two variables determine the initial state cursor visibility and lock state. There are two other aspects that are controlled: time scale and pause state, but these are assumed to be 1 and false on initialization, since that will be the case 99% of the time.⁴

[Header("Callbacks")]
[Tooltip("Callback for when the game goes from unpaused to paused.")]
public UnityEvent OnPause;
[Tooltip("Callback for when the game goes from paused to unpaused.")]
public UnityEvent OnUnpause;

As I mentioned up above, being paused is a construct I use in my games to do things like blocking scripts from accessing input events, which can happen even when the time scale is zero. In previous iterations this was just a BoolReference (a scriptable object that only maintains a bool), but for the public release I replaced this with Unity event callbacks.

private Stack<IMenu> _menuStack = new Stack<IMenu>();
private Stack<MenuAttributes> _cachedMenuAttributes 
    = new Stack<MenuAttributes>();

private bool _paused;

void Awake() {
    Shared = this;
    Cursor.lockState = DefaultLockMode;
    Cursor.visible = CursorVisible;
    _paused = false;
    OnUnpause?.Invoke();
}

In awake we’re setting the variables to the defaults that were established by the user in the inspector above. Behavior will be misleading if running in the inspector, as it doesn’t properly honor cursor lock state. Unpause is invoked to correctly set the initial state of all pause variables. Generally I like callbacks to send out their initial state on initialization, but that’s not a universal opinion.

public bool PushAndShowMenu(IMenu menu) {
    if (menu == null) {
        Debug.LogWarning("Attempting to push a null menu!");
        return false;
    } else if (_menuStack.Contains(menu)) {
        Debug.LogWarning("Attempting to push menu already in stack.");
        return false;
    }

    CacheCurrentMenuAttributes();
    SetTopStatusOfTopOfStack(false);
    _menuStack.Push(menu);
    ApplyMenuAttributes(menu.GetMenuAttributes());
    menu.SetMenuUp(true);
    menu.SetMenuOnTop(true);
    return true;
}

public bool PopAndCloseMenu(IMenu menu) {
    if (_menuStack.Count == 0) {
        Debug.LogWarning("Attempting to pop menu but stack is empty!");
        return false;
    } else if (_menuStack.Peek() != menu) {
        Debug.LogWarning("Attempting to pop menu not on top of stack!");
        return false;
    }
    PopAndApplyMenuAttributes();
    if (_menuStack.Count > 0) {
        IMenu top = _menuStack.Pop();
        top.SetMenuOnTop(false);
        top.SetMenuUp(false);
        SetTopStatusOfTopOfStack(true);
    }
    return true;
}

This is the meat of MenuStack’s behavior. You can push a menu onto the stack and pop a menu off of the stack. To avoid strangeness when users are interacting, re-adding a menu or removing a non-active menu is prohibited.

IMenu’s callbacks are invoked in a specific order. Note that telling a menu will be told that it’s closed after it’s told that it’s not on top, and it will be told that it is open before it’s told that it is on top. By having the top state callback encapsulated by the open state callback, it’s simpler to write code that handles those two cases gracefully. See the Android activity lifecycle for a similar flow.

Similarly, a menu will only be told that it is on top after another menu has been told that it is no longer on top. This prevents cases where two separate menus are fighting over some bit of shared state. Imagine that you have a reference to the active menu in some shared class, OtherMenuStack. If your NotOnTop callback set OtherMenuStack.activeMenu = null and the OnTop set it to OtherMenuStack.activeMenu = this, performing these in a different order would cause you to be unable to set this bit of shared state in a callback.⁵

public void ToggleMenu(IMenu menu) {
    if (!_menuStack.Contains(menu)) {
        PushAndShowMenu(menu);
    } else if (_menuStack.Count > 0 && _menuStack.Peek() == menu) {
        PopAndCloseMenu(menu);
    }
}

public void PopAndPushNewMenu(IMenu current, IMenu newMenu) {
    PopAndCloseMenu(current);
    PushAndShowMenu(newMenu);
}

Some simple helpers to simplify some behaviors I think will be common. Note that I’m once again being opinionated about when you can open or close a menu, although it’s overkill to enforce it again here.

    private void SetTopStatusOfTopOfStack(bool newStatus) {
        if (_menuStack.Count == 0) return;
        _menuStack.Peek().SetMenuOnTop(newStatus);
    }

    public int StackSize() {
        return _menuStack.Count;
    }

    public bool IsMenuInStack(IMenu menu) {
        return _menuStack.Contains(menu);
    }

    public bool IsMenuAtTop(IMenu menu) {
        return _menuStack.Peek() == menu;
    }

    public bool IsMenuUp(IMenu thisMenu) {
        foreach(IMenu menu in _menuStack) {
            if (thisMenu == menu) return true;
        }
        return false;
    }

A variety of helper functions, mostly to provide information about the stack externally without providing access to the stack object itself. Why? Because you can’t trust developers not to mess with it and break things. Or rely on altering it as part of their workflow.⁶

    void UpdatePaused(bool newState) {
        if (newState != _paused) {
            _paused = newState;
            if (_paused) OnPause?.Invoke();
            else OnUnpause?.Invoke();
        }
    }

Here’s how you can send out Unity events. Note that the question mark in OnPause?.Invoke() is a short-hand. It’s equivalent to writing this:

if (OnPause != null) OnPause.Invoke();

The ? says if this isn’t null, do the following action. If it is, return null. This is called a null conditional operator.

    void CacheCurrentMenuAttributes() {
        MenuAttributes attributes = new MenuAttributes();
        attributes.cursorLockMode = Cursor.lockState;
        attributes.cursorVisible = Cursor.visible;
        attributes.timeScale = Time.timeScale;
        attributes.pauseGame = _paused;
        _cachedMenuAttributes.Push(attributes);
    }

    void PopAndApplyMenuAttributes() {
        if (_cachedMenuAttributes.Count == 0) {
            Debug.LogWarning(
                "Attempting to pop menu attributes but stack is empty!");
            return;
        }
        MenuAttributes attributes = _cachedMenuAttributes.Pop();
        ApplyMenuAttributes(attributes);
    }

    void ApplyMenuAttributes(MenuAttributes attributes) {
        if (!DisableCursorManagement) {
            Cursor.lockState = attributes.cursorLockMode;
            Cursor.visible = attributes.cursorVisible;
        }
        if (!DisableTimeManagement && attributes.timeScale >= 0) {
            Time.timeScale = attributes.timeScale;
        }
        UpdatePaused(attributes.pauseGame);
    }
}

This is where we get to the heart of the issue posed at the start: how do we manage time scale, paused state, and cursor variables. It’s pretty simple. When a menu is pushed, we cache the current state in a stack,⁷ and restore that state when the menu opened over it is closed. As long as every one involved in the chain is a good citizen, which they will be as they’re forced to as a part of MenuStack, menus will have the state restored to them that they left in.

The Hook

All of this is well and good when you have control over the menus. After all, it’s easy enough to set the behavior of code that you control. What if some library exposes some arcane combination of code that displays a menu without using a canvas. Certainly they won’t implement MenuAttributes by themselves.

Enter MenuHook:

public class MenuHook : MonoBehaviour, IMenu {
    [Header("General")]
    public bool ShowOnStart = false;
    // The canvas element. If this is set, this script will automatically 
    // hide/show the canvas when appropriate.
    public Canvas Canvas;

    [Header("Menu Attributes")]
    public CursorLockMode CursorLockMode = CursorLockMode.None;
    public bool CursorVisible = true;
    public bool PausesGame = true;
    // Time scale to use. If negative, it will use the existing time scale.
    public float TimeScale = 0;

    [Header("Hooks")]
    public UnityEvent OnMenuOpen;
    public UnityEvent OnMenuClose;
    public UnityEvent OnMenuTop;
    public UnityEvent OnMenuNotTop;

This is the meat of MenuHook: It provides ways to specify cursor lock mode, whether the cursor is visible, and all the other attributes that the menu supports. The menu attributes are for the convenience of the developer if they’re hooking in a bit of their code so that they don’t need to reinvent the wheel themselves. If the menu being hooked into sets these values themselves, they’ll just overwrite the values here. And if that menu calls to MenuStack to push a menu,⁸ the attributes cached are the current ones, not the ones set here.

The hooks allow easily hooking into the menu’s show and hide methods, so your menu can easily do a PushAndShowMenu on this MenuHook and get the correct behavior.

    void Awake() {
        if (Canvas != null) Canvas.enabled = false;
    }

    void Start() {
        if (ShowOnStart) {
            PushMenu();
        }
    }

    public MenuAttributes GetMenuAttributes() {
        MenuAttributes attributes = new MenuAttributes {
            cursorLockMode = CursorLockMode,
            cursorVisible = CursorVisible,
            pauseGame = PausesGame,
            timeScale = TimeScale
        };
        return attributes;
    }

    public void SetMenuUp(bool newUp) {
        if (Canvas != null) Canvas.enabled = newUp;
        if (newUp) OnMenuOpen?.Invoke();
        else OnMenuClose?.Invoke();
    }

    public void SetMenuOnTop(bool newOnTop) {
        if (newOnTop) OnMenuTop?.Invoke();
        else OnMenuNotTop?.Invoke();
    }

    public void PushMenu() {
        MenuStack.Shared.PushAndShowMenu(this);
    }

    public void PopMenu() {
        MenuStack.Shared.PopAndCloseMenu(this);
    }
}

The rest of the class is providing convenience methods and responding to MenuStack’s callbacks to call out to the callbacks hooked up in MenuHook. It’s as easy as that.

Conclusion

With that, we’ve set up a functioning menu system that removes most of the boilerplate of menus on top of menus without preventing ourselves from using menus from outside of our control. An iterated-upon version of these classes (along with samples as well as convenience methods and scripts) is included in my github project here.

Feel free to request a feature or send a pull request my way. And if you do end up using any of this stuff, let me know, I’d love to hear it.