I stand corrected on the marble reference. Thank you, Masato, that is fascinating!

Here is one more example of the actual phrase being used (my emphasis). This is from an article in 1997 which tells of a murder trial in 1906:

News Herald

She told of hearing harsh words between the Judge and his son, Will. The Judge wanted to know more about the "$600" and was heard to say: "Me and your mother ain't going to be pulled down by you this way. We didn't have anybody to build us up. We came up by our muscle in a little log cabin the size of a fowl house. This $600 has got to come." His son replied: "Don't let it bother you. It will all come to taw at the right time."