The way the government allocated funding had some "room for improvement," Mr Osborne admitted.
The Chancellor was giving evidence to the MPs during a hearing by the House of Commons Treasury Committee on Thursday (12 December).
The committee is taking evidence on the government's 2013 Autumn Statement, outlined by Mr Osborne earlier this month.
The question of apportioning public expenditure in areas such as health was "complicated," he told MPs at the hearing.
Mr Osborne said: "The truth is that our urban areas do get relatively more public spending, but they tend to also have much greater health and deprivation problems."
There was "always a debate" about how much money per head of population his own Cheshire constituency received compared to neighbouring Manchester, continued Mr Osborne.
He added: "I think there is room for improvement in some of these formulas, but fundamentally we make a decision as a country that we spend more money in the most deprived areas."
Tory MP Jesse Norman (Hereford and South Herefordshire) then asked Mr Osborne whether this was still the case when those deprived areas were sometimes rural, rather than urban.
Mr Osborne replied: "One of the things that these formulas have not been very good at tackling is pockets of rural deprivation."
The Chancellor then moved on to the national funding formula for schools, which he said the government was examining.
"You could have two schools with the same number of kids on free school meals, which is not a bad indicator of deprivation, and yet because one is in a rural area and one is in an urban area, the rural school gets much less.
"That is something we are looking at in the national funding formula for schools.
"Of course, as I say, there is a broader debate to be had, but I think you are still going to end up with a situation where you are concentrating resources in areas of greater deprivation."
The Chancellor's comments follow a Rural Fair Share campaign led by Yorkshire MP Graham Stuart and supported by the Rural Services Network.
This autumn, more than 100 petitions calling for fairer rural funding were presented to the government by MPs from 119 rural constituencies.
It is thought to have been one of the largest such petitions ever gathered.
Currently, urban councils receive 50% more government grant per head than local authorities in rural areas.
This is despite the fact that rural residents pay on average £85 per head more in council tax and many public services are more expensive to deliver for sparsely-populated councils.
The petitions - totalling some 20,000 public signatures - oppose government plans to lock-in this inequality for the next six years.
They also call for the funding gap to be reduced by at least 10% by 2020.
A full transcript of Mr Osborne's evidence session before the Treasury select committee can be seen here.