This study investigated farm level challenges and factors affecting the sources of income among pepper farmers in Kaduna and Kano States, Nigeria. A simple random sampling design was utilized to select 200 pepper growers. Primary data were employed utilizing a well-structured questionnaire. Data were evaluated utilizing descriptive statistics, Gini-Coefficient, Kendall’s coefficient of concordance, and Multinomial Logit model. The results show that the mean age of pepper farmers was 46 years, with a n average of 13 years of attendance in school education. They are smallholder farmers with an average of 1.27 ha of pepper farms. Approximately, 70% (140) of pepper farmers belong to high income inequality group, while 30% (60) belongs to low income inequality group. The main sources of income include farm income (34.04%), non-farm income (27.66%), and off-farm income (25.53%). The significant factors affecting the sources of farm income among pepper growers include education (P < 0.01), experience (P < 0.05), access to market (P < 0.01) and access to inputs such as fertilizer usage (P < 0.01). The significant factors affecting the sources of non-farm income include age (P < 0.01), cooperative membership (P < 0.10), and access to market (P < 0.01). The study recommended improved infrastructures such as better roads, irrigation systems, and improved market access. Furthermore, improved access to credit, and providing fertilizers, pesticides, and improved seeds at subsidized rate can reduce farmers’ costs and increase productivity.