In French, forming the feminine of nouns that denote people (particularly professions and roles) follows specific patterns depending on the masculine ending. For masculine nouns ending in -teur, the feminine is typically formed by changing the ending to -trice when the masculine form derives from a Latin root ending in -tor.
The word acteur (actor) follows this pattern. Its Latin root is actor, so the feminine is formed by replacing -teur with -trice, giving actrice (actress).
The other proposed forms do not follow any standard French feminine formation rule:
acteuse would follow the -eur to -euse pattern (as in danseur/danseuse), but this pattern applies to nouns derived from French verbs, not Latin roots in -tor.
acteurre and actere do not correspond to any French morphological rule.
Other common examples of the -teur/-trice pattern include directeur/directrice, instituteur/institutrice, and spectateur/spectatrice.