Generally they stop digging when the funding runs out. Although they can also take some reasonable guesses as to whether an area is likely to be worth the time. If they’re digging in an established area and they see signs that they’ve reached the edge of the habitation, either in area of in time, there probably isn’t much point in continuing to dig. There *might* be–there could be an adjacent settlement or one that existed earlier in time–but the chances are lower. It ends up being a bit of a judgment call.
When establishing a new dig area, they typically conduct a number of different surveys to try to predict where they’ll most likely find something. Satellite photos have become very popular for this but other methods exist, including picking an area and digging some random test holes.
Latest Answers