AI in Production: What Does “It Works” Mean?

Would love your take on this Data Wire piece with MikeRusso and Ricard Galvez on moving AI from pilot into production in capital markets.

One question from Richard really stuck with me: what does “it works” actually mean?

A model can pass its evaluation and still not make the work better, faster, or less expensive.

Pilots measure whether the model performs. Production has to measure whether the work improved.

That second test is harder, and it often does not have a clear owner.

So I’m curious: on your team, what does “it works” mean for AI?