The development of edge computing, Internet-of-Things (IoT), and AI-driven services involves increasing volumes of sensitive data—health records, industrial sensor readings, personal information—at the network edge. Regulatory frameworks such as the GDPR, the European Health Data Space, and the EU AI Act impose strict requirements on such processing. However, privacy-enhancing technologies face trade-offs between utility, privacy guarantees, and computational cost that limit their practical use and uptake.

This thesis investigates how these trade-offs can be addressed through encrypted machine learning (ML) inference, architecture co-design for efficient encrypted computation, and privacy-preserving model training. The six appended papers, complemented by additional publications, address these trade-offs across health data infrastructure, edge data center management, and privacy regulation.

First, fully homomorphic encryption (FHE) is applied to data processing and control pipelines to investigate practically feasible encrypted computation on health data and encrypted remote monitoring and control of edge data center systems for lightweight ML tasks. Two leading encryption schemes—CKKS and TFHE—are evaluated, showing that both data and algorithm confidentiality can be achieved simultaneously, with encrypted execution times ranging from milliseconds to seconds.

Second, the computational bottleneck of neural network inference under FHE is addressed through architecture co-design. A novel mechanism is proposed—the Inhibitor—which replaces the multiplications and softmax activations of conventional gated RNNs and Transformer attention with encryption-friendly addition and ReLU. Experiments demonstrate 3–6× speedup for encrypted inference, and knowledge distillation produces a compact Inhibitor language model competitive on corresponding NLP benchmarks.

Third, methods are developed for training on distributed or sensitive data without compromising privacy. A local conditioning approach for heterogeneous federated learning is introduced, where locally computed statistics replace cross-client coordination, enabling scalable personalization without revealing distributional information or adding communication overhead. A proactive defense against training data memorization is also proposed, reducing the log-likelihood ratio for membership identification by an order of magnitude at the chosen audit precision, while maintaining model accuracy.

Taken together, the results suggest that privacy-preserving ML can be made more feasible for resource-constrained settings, including latency-sensitive edge deployments, helping to narrow the gap between regulatory requirements and computational cost. The evaluation relies on established benchmarks and controlled numerical experiments; validation in production deployments remains future work.