Antibiotic resistance is a growing concern in medicine and raises the need to develop and design new drug molecules that can efficiently inhibit bacterial replication. Spurring the passive uptake of the drug molecules is an obvious solution. However our limited understanding of drug-membrane interactions due to the presence of an overwhelming variety of lipids constituting cellular membranes and the lack of facile tools to probe the bio-physical interactions between drugs and lipids imposes a major challenge towards developing new drug molecules that can enter the cell via passive diffusion. Here, we used a label-free micro-fluidic platform combined with giant unilamellar lipid vesicles to investigate the permeability of membranes containing mixtures of DOPE and DOPG in DOPC, leading to a label-free measurement of passive membrane-permeability of autofluorescent antibiotics. A fluoroquinolone drug, norfloxacin was used as a case study. Our results indicate that the diffusion of norfloxacin is strongly dependent on the lipid composition which is not expected from the traditional octanol-lipid partition co-efficient assay. The anionic lipid, DOPG, slows the diffusion process whereas the diffusion across liposomes containing DOPE increases with higher DOPE concentration. Our findings emphasise the need to investigate drug-membrane interactions with focus on the specificity of drugs to lipids for efficient drug delivery, drug encapsulation and targeted drug-delivery.