In a small pot over medium-low heat, simmer the cream, milk, sugar, salt, and lavender until the sugar dissolves, about 5 minutes. Reduce heat to low and let stand 10 minutes more to infuse the lavender. Remove from the heat.
In a medium bowl, whisk the yolks for 1 minute. Slowly whisk 1/3 of the hot cream mixture into the yolks, then whisk the yolk mixture back into the pot with the cream mixture.
Return the pot to medium-low heat, and cook while whisking to 170F. The mixture will slightly thicken, and should coat back of a spoon.
Strain the mixture through a fine mesh sieve into a clean bowl and allow to cool to room temperature. You may add food coloring at this stage-- be sure to stir REALLY well if you use gel food coloring. Cover and chill in the refrigerator at least 4 hours, or up to overnight.