Fruit dropping off can be caused by many things:
Solution: Determine the cause of the problem.
If flowering is early and coincides with cold weather, you may be able to remove those flowers to induce a second flowering.
Avoid insecticide applications during flowering
Adjust boron application rates