It wasn't agriculture. It was agricultural surplus. When they finally nailed agriculture down well enough that it wasn't necessary for everybody to work in the fields, some people were able to leave the fields and become specialists. Instead of a farmer's wife making crappy baskets, he could trade some of his surplus grain for good baskets made by someone who knew what he was doing. Eventually, barter became too cumbersome and money was invented. That's when it really started to go wrong because money only works if it has the force of power behind it. How much is a stack of 10,000 ruble notes from the Societ Union worth? Nothing, because the force of power that once backed them is no more. So, for the convenience of having money, people willingly let themselves be dictated to by whoever was the biggest bully.