Be sure to use a candy thermometer when deep-frying to ensure the temperature is right. Also cook the cauliflower in batches, so as not to crowd it, which would lower the temperature and cause the cauliflower to absorb oil rather than achieve the desired crispiness.
In a large bowl, beat the eggs and toss in the cauliflower florets to coat well. Shake off any excess egg coating and place the florets on a plate.
Sprinkle the florets evenly with almond flour and chili powder (if using).
Fill a deep frying pan or wok with canola oil to a depth of 2 inches; heat to 350ºF.
Fry the florets in batches for about 2 minutes each, placing them on paper towels to soak up excess oil. Transfer to a serving plate.
To serve, drizzle cauliflower with fish sauce and lime juice and top with chopped scallions.