Abdulla Khaybullayev scored the only goal of the contest in the 25th minute as Azerbaijan held on to claim three points and move to the top of Group D2.
Khaybullayev gives Azerbaijan the lead
Azerbaijan started positively and broke the deadlock midway through the first half.
Khaybullayev found the net in the 25th minute after being set up by Toral Bayramov, giving the hosts a 1-0 advantage.
Lithuania were unable to respond before half-time, despite enjoying more possession as they attempted to find a way back into the contest.
Azerbaijan withstand Lithuania pressure
Lithuania increased the pressure after the restart and created opportunities to equalise.
Azerbaijan goalkeeper Emil Balayev, however, produced a perfect performance between the posts, making three saves to preserve his clean sheet.
The visitors finished with 62 per cent possession and 13 attempts, but only three of those efforts were on target. Azerbaijan registered seven shots on target from 13 attempts.
Late drama in Baku
Azerbaijan had a scare in the second half when Khaybullayev was initially shown a second yellow card in the 51st minute.
The decision was subsequently overturned following a VAR review, allowing the goalscorer to continue before being substituted later in the match.
Lithuania were reduced to 10 men deep into stoppage time when Armandas Kučys was shown a straight red card in the seventh minute of added time.
Azerbaijan nevertheless held firm to secure their first win of the group stage.
Azerbaijan move top of Group D2
The victory took Azerbaijan to five points from three matches, moving them above Lithuania, who remain on four points.
Azerbaijan had drawn their opening two matches against Lithuania and Liechtenstein before finally securing their first win.
The two teams will now go their separate ways, with Azerbaijan travelling to face Liechtenstein in November, while Lithuania will host Liechtenstein.
Match summary
• Azerbaijan: Abdulla Khaybullayev 25'
• Lithuania: No goals
Final score: Azerbaijan 1-0 Lithuania