A Labor victory would be its first in Bega, which has been held by the Liberals since its creation in 1988 and by its retiring member Andrew Constance since 2003.
Early tallies in the south coast seat on Saturday night gave a swing of more than 14 per cent to the ALP's Michael Holland over the Liberal Party's Fiona Kotvojs.
Dr Holland says the "incredible swing" to Labor in Bega is "a reflection of the community's response to the current government".
"It's also been a plebiscite ... to wanting a local member who will represent them with integrity and honesty," Dr Holland said on ABC News.
With votes from just under a third of the electorate counted as of 10.45pm on Saturday, Mr Holland had received 57.1 per cent of two-candidate preferred votes, compared to Ms Kotvojs' 42.9 per cent, according to mid-tally results posted by the NSW Electoral Commission.
The Liberals went into the by-election with a margin of 6.9 per cent.
A loss in Bega would push the government further into minority and would force it to rely more heavily on the votes of independent or minor party MPs.
Labor candidate Jason Yat-Sen Li also looks set to retain Strathfield, leading Liberal candidate Bridget Sakr 54.4 per cent to 45.6 per cent with about 37 per cent of the electorate's votes counted.
Premier Dominic Perrottet insisted Labor's vote had gone backwards in the electorate and vowed Ms Sakr would "win the election next year".
The Nationals are on track to hold former deputy premier John Barilaro's seat of Monaro despite a swing of more than six per cent.
With about 44 per cent of the vote counted, Nichole Overall led Labor's Bryce Wilson 55 per cent to 45 per cent.
The premier applauded his party's "phenomenal" results despite the big swings against the government in early counting, including an 18 per cent swing in Willoughby against the Liberals' Tim James.
He faced independent Larissa Penn in the blue-ribbon seat on Sydney's north shore.
Two-candidate preferred results for Willoughby, which Labor did not contest, are not yet available.
Mr James has so far secured about 43.4 per cent of first preference votes, trailed by Ms Penn with 32.1 per cent, with just over 40 per cent of the electorate's votes counted.
The by-lections were viewed as a test for the premier and his government's handling of the state's wave of the Omicron variant.
The stakes were also high for Labor leader Chris Minns, who has pitched himself as the man to make Labor a real election chance after more than a decade in opposition.
Mr Minns said voters had sent the premier a message on Saturday.
"The government needs to change direction. It must listen to expert advice. It must have commonsense solutions to difficult problems," Mr Minns said.
Close to 40 per cent of constituents voted early or via post, so results are not expected to be finalised for some time.
The NSW Electoral Commission says no votes will be counted on Sunday.
Counting will resume on Monday, and postal votes will be counted from next weekend, with final results not expected until later this month.
AAP with The Project