Biffa planned to use a business park at Llansamlet to process non-hazardous waste which would otherwise go to landfill.
Planning officers had recommended approval saying there was no significant impact on residents.
But thousands of people have opposed the plans, with Extinction Rebellion staging a protest at Biffa premises.
More than 2,500 objection letters were sent in during a public consultation on the incinerator, intended to replace an unused warehouse on Swansea Enterprise Park.
Biffa said it would process 21,000 tonnes of non-hazardous waste that would otherwise go to landfill in Merthyr Tydfil, creating about 10 construction jobs, and 15 full-time jobs.
A report by Swansea Council officers to councillors said the location was “acceptable in principle within an established commercial and industrial area”.
Despite “significant concerns” about air quality and health, both the local health board and Swansea council’s Pollution Control Division felt it would not have “any significant impacts” on the environment or human health.
The company insisted pollution levels would be below those set by UK air quality standards.
Swansea Council leader Rob Stewart was among those who spoke out against the plan at the committee meeting on Tuesday, saying support for the incinerator would be a “retrograde step”, according to the Local Democracy Reporting Service.
The committee voted unanimously against the proposal.
Jules Woodell, who took part in the Extinction Rebellion protest at Biffa’s Swansea depot on Tuesday morning, said: “We’ve got to act now and act quickly and this is a step in the wrong direction – burning 20,000 tonnes of waste a year and putting carbon in the air.”
He added: “The Welsh Government has declared a climate emergency and has the Future Generations Act; this flies in the face of all of that.
“We hope the protest will send a strong message to councillors in the meeting today, that the people of Swansea and residents are strongly opposed to this.
“They have to think of future generations and the bigger picture in how they deal with waste.”
The Welsh Government said the decision over the incinerator was one for the local authority.
A spokeswoman for Biffa said: “We’re disappointed with today’s decision and we will be reviewing our position and next steps over coming days.”