Fisher Investments founder Ken Fisher created a public relations firestorm for his firm last week after attendees of an industry conference took to Twitter to complain that comments he made from the stage were vulgar and offensive.

The backlash prompted the conference host to ban him from future events and forced Fisher to apologize publicly and create a “diversity and inclusion task force” at his firm. But Fisher’s apologies have not appeased the officials responsible for the state of Michigan’s pension fund, which Friday withdrew $600 million from the firm.

Fisher has spent nearly four decades pitching his fee-only practice as a champion of investors of all sizes. The Camas, Wash.-based firm has also spent heavily on high-profile ad campaigns to build a practice that its website says represented $112 billion in client money at the end of September. But in the era of #metoo, can his comments, which peers lambasted as “offensive” and “horrible,” kill the Fisher Investments brand?