flutter Toggle Button uses always the default color

I have a problem with my toggleButton and it is not clear where the problem is. When I run the below code the toggle buttons appear in the desired location and are also payable as desired. However, they are always only in the default color (blue background when selected, no background when not selected) and not as defined in the ToggleButtons. I have already tried a few things, like defining the lists as List<List<bool>> and List<List<Widget>>, which I found as a solution in other questions. But then I get errors on the lists generated in initState(), in the setState() method, on isSelected and children.
Thanks for your help.

Code:

import 'package:calendar_vertical/provider/toggle_provider.dart';
import 'package:calendar_vertical/provider/user_model.dart';
import 'package:calendar_vertical/widgets/user_item.dart';
import 'package:flutter/material.dart';
import 'package:provider/provider.dart';

class ToggleButtonUsers extends StatefulWidget {
  @override
  _ToggleButtonUsersState createState() => _ToggleButtonUsersState();
}

class _ToggleButtonUsersState extends State<ToggleButtonUsers> {
  bool vertical = false;
  List<bool> selectedUser = [];
  //final List<Widget> children = [];
  List<Widget> selectedChildren = [];

  @override
  void initState() {
    final userModelData = Provider.of<UserModel>(context, listen: false);
    final userModel = userModelData.items;
    final toggleModelData = Provider.of<ToggleProvider>(context, listen: false);
    final toggleModel = toggleModelData.items;

    for (var index = 0; index < toggleModel.length; index += 1) {
      selectedUser = List.generate(
        toggleModel.length,
        (index) => index == 0 ? true : false,
      );

      selectedChildren = List.generate(
        toggleModel.length,
        (index) => UserItem(
          userModel[index].id,
          userModel[index].name,
          userModel[index].image,
          userModel[index].color,
        ),
      );
    }
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return Padding(
      padding: const EdgeInsets.all(8.0),
      child: ToggleButtons(
        constraints: const BoxConstraints(
          minHeight: 40.0,
          maxHeight: 45,
          minWidth: 80.0,
          maxWidth: 85,
        ),
        selectedBorderColor: Colors.green,
        disabledBorderColor: Colors.red,
        direction: vertical ? Axis.vertical : Axis.horizontal,
        onPressed: (int index) {
          setState(() {
            selectedUser[index] = !selectedUser[index];
          });
          debugPrint('selected User ist nun: ' + selectedChildren.toString());
        },
        children: selectedChildren,
        isSelected: selectedUser,
      ),
    );
  }
}

you can use other color options like fillColor, selectedColor, disabledColor, fillColor, focusColor, highlightColor, hoverColor, splashColor.

Use fillColor if you want to change the background color or selectColor if you want to change the color of the contents inside.

But, The disabledColor and disableddBorderColor determine the color when the button is disabled. This property only applies if the button is used without an onPressed callback. This means that it is applied when the button cannot be pressed (disabled).

Also, focusColor Determines the background color when the button is focused. Applies when keyboard navigation is enabled or when the button is programmatically focused.