Objective To develop a deep learning-based tongue image classification system for psoriasis to improve the objective differentiation between the patterns of Spleen Deficiency with Dampness Retention （SDDR） and Blood Stasis （BS）. Methods A total of 981 tongue images from psoriasis patients diagnosed with SDDR or BS were collected. An improved U-Net model， featuring a ResNet-34 encoder， bilinear interpolation upsampling， and optimized skip connections， was employed for automatic tongue region segmentation. Macenko color normalization and the Albumentations library were applied for data augmentation to mitigate variances from imaging devices and lighting conditions. A two-stage framework was constructed： the first stage precisely extracted the tongue body region， while the second stage utilized a Hybrid Model integrating EfficientNet-B3 and Swin-Tiny architectures for pattern classification. A cross-modal multi-head attention mechanism was introduced to fuse local textural and global structural features. Results The improved U-Net achieved superior performance in tongue segmentation， with a Dice coefficient of 0.98 and an IoU of 0.89， significantly outperforming the original U-Net （Dice 0.85）. For pattern classification， the Hybrid Model demonstrated the best overall performance， achieving a 5-fold cross-validation mean accuracy of 0.9816 and a mean AUC of 0.9993. The F1-score was significantly higher than those of individual models. Macenko normalization contributed to an 8.3% increase in F1-score. The inference time per image was 38 ms on an A10 GPU， meeting the requirement for clinical real-time application. Conclusion The constructed two-stage tongue image classification model effectively and accurately distinguishes between SDDR and BS patterns in psoriasis， significantly enhancing the objectivity of tongue diagnosis. It provides a reliable tool for pattern differentiation in Traditional Chinese Medicine and shows promising potential for clinical application.