This idiomatic expression is commonly used in an informal manner. It can mean there is a much more pressing issue to attend to or a higher-valued result or target to reach. This expression did not catch on until John Evelyn used it in his memoirs during the 17th century.
Additional Examples:
Instead of worrying about petty thieves, the police have bigger fish to fry, as four bank heists were carried out in the same city.
That’s not my concern at the moment. My business is in disarray, so I’ve bigger fish to fry.