This is a common misconception.  While the Group Manager is added by default to the access when you view the task, the access is not actually set until you view each groups access.  Simply navigating to this task by going to Core > Security > Group Page Access and viewing each group type will set the default access giving the Group Manager the access they need but the access is not set until the tasks are actually view in the software.

To sum it up, the default group page access is not set until the manager actually looks at the Group Page Access task for each group type