仿真环境代码如何在生产环境执行，教程SPU举例

[gphper]

学习SPU这块代码时使用的是 仿真模式，代码如下

import secretflow as sf

sf.init(['alice', 'bob', 'carol', 'dave'], address='local')

aby3_config = sf.utils.testing.cluster_def(parties=['alice', 'bob', 'carol'])

spu_device = sf.SPU(aby3_config)

alice, dave = sf.PYU('alice'), sf.PYU('dave')

def deduce_from_account(bank_account, amount):
    new_bank_account = []

    for account in bank_account:
        account['deposit'] = account['deposit'] - amount
        new_bank_account.append(account)

    return new_bank_account

bank_account = [{'id': 12345, 'deposit': 1000.25}, {'id': 12345, 'deposit': 100000.25}]
bank_account_spu = sf.to(alice, bank_account).to(spu_device)

def debit_amount():
    return 10


debit_amount_pyu = alice(debit_amount)()
debit_amount_spu = debit_amount_pyu.to(spu_device)

new_bank_account_spu = spu_device(deduce_from_account)(
    bank_account_spu, debit_amount_spu
)

print(sf.reveal(new_bank_account_spu))

代码中把alice和bob两方的数据是放到一起的，但是隐私计算的宗旨应该是可用不可见，像上面的代码这样，不就违背这个原则了吗，如果在生成环境中使用的话，Alice和Bob的代码开发应该怎么写，其实这里可以着重看下 隐语的编程思想 这篇文档，看完就懂了。我这里使用docker 模拟搭建了生产环境，有问题的兄弟可以自己动手试试

在宿主机创建目录
|--alice
|----index.py
|--bob
|----index.py
|--docker-compose.yml

docker-compose.yml 文件

services:
  alice:
    image: secretflow/secretflow-anolis8:latest
    pull_policy: if_not_present
    platform: linux/amd64
    environment:
      - SELF_PARTY=alice
    volumes:
      - ./alice:/home/workspace
    tty: true  

  bob:
    image: secretflow/secretflow-anolis8:latest
    pull_policy: if_not_present
    platform: linux/amd64
    environment:
      - SELF_PARTY=bob
    volumes:
      - ./bob:/home/workspace
    tty: true  

alice/index.py

import secretflow as sf
import spu

network_conf = {
    "parties": {
        "alice": {
            "address": "alice:8000",
        },
        "bob": {
            "address": "bob:8000",
        },
    },
    'self_party': 'alice'
}

sf.init(address='127.0.0.1:6379', cluster_config=network_conf)

alice, bob = sf.PYU("alice"), sf.PYU("bob")
spu_conf = {
    "nodes": [
        {
            "party": "alice",
            "address": "alice:8001",
            "listen_addr": "0.0.0.0:8001",
        },
        {
            "party": "bob",
            "address": "bob:8001",
            "listen_addr": "0.0.0.0:8001",
        },
    ],
    "runtime_config": {
        "protocol": spu.spu_pb2.SEMI2K,
        "field": spu.spu_pb2.FM128,
        "sigmoid_mode": spu.spu_pb2.RuntimeConfig.SIGMOID_REAL,
    },
}
spu_device = sf.SPU(cluster_def=spu_conf)

def deduce_from_account(bank_account, amount):
    new_bank_account = []

    for account in bank_account:
        account['deposit'] = account['deposit'] - amount
        new_bank_account.append(account)

    return new_bank_account

bank_account = [{'id': 12345, 'deposit': 1000.25}, {'id': 12345, 'deposit': 100000.25}]
bank_account_spu = sf.to(alice, bank_account).to(spu_device)

def debit_amount():
    return 0


debit_amount_pyu = bob(debit_amount)()
debit_amount_spu = debit_amount_pyu.to(spu_device)

new_bank_account_spu = spu_device(deduce_from_account)(
    bank_account_spu, debit_amount_spu
)

print(sf.reveal(new_bank_account_spu))

bob/index.py

import secretflow as sf
import spu

network_conf = {
    "parties": {
        "alice": {
            "address": "alice:8000",
        },
        "bob": {
            "address": "bob:8000",
        },
    },
    'self_party': 'bob'
}

sf.init(address='127.0.0.1:6379', cluster_config=network_conf)

alice, bob = sf.PYU("alice"), sf.PYU("bob")
spu_conf = {
    "nodes": [
        {
            "party": "alice",
            "address": "alice:8001",
            "listen_addr": "0.0.0.0:8001",
        },
        {
            "party": "bob",
            "address": "bob:8001",
            "listen_addr": "0.0.0.0:8001",
        },
    ],
    "runtime_config": {
        "protocol": spu.spu_pb2.SEMI2K,
        "field": spu.spu_pb2.FM128,
        "sigmoid_mode": spu.spu_pb2.RuntimeConfig.SIGMOID_REAL,
    },
}
spu_device = sf.SPU(cluster_def=spu_conf)

def deduce_from_account(bank_account, amount):
    new_bank_account = []

    for account in bank_account:
        account['deposit'] = account['deposit'] - amount
        new_bank_account.append(account)

    return new_bank_account

bank_account = []
bank_account_spu = sf.to(alice, bank_account).to(spu_device)

def debit_amount():
    return 10


debit_amount_pyu = bob(debit_amount)()
debit_amount_spu = debit_amount_pyu.to(spu_device)

new_bank_account_spu = spu_device(deduce_from_account)(
    bank_account_spu, debit_amount_spu
)

print(sf.reveal(new_bank_account_spu))

执行命令
docker-compose up -d

然后进入到容器中分别在容器中开启ray集群

ray start --head --port=6379 --include-dashboard=False --disable-usage-stats --resources='{"alice": 1}'
ray start --head --port=6379 --include-dashboard=False --disable-usage-stats --resources='{"bob": 1}'

分别在两个容器的 /data/workspace 目录下，执行 pytohn index.py 等待执行完成，最后会得到如下结果

2025-04-12 06:00:04,338 INFO worker.py:1540 -- Connecting to existing Ray cluster at address: 172.
18.0.2:6379...
2025-04-12 06:00:04,348 INFO worker.py:1724 -- Connected to Ray cluster.
2025-04-12 06:00:04.369 INFO api.py:233 [alice] -- [Anonymous_job] Started rayfed with {'CLUSTER_A
DDRESSES': {'alice': 'alice:8000', 'bob': 'bob:8000'}, 'CURRENT_PARTY_NAME': 'alice', 'TLS_CONFIG'
: {}}
2025-04-12 06:00:05.544 INFO barriers.py:284 [alice] -- [Anonymous_job] Succeeded to create receiv
er proxy actor.
(ReceiverProxyActor pid=66123) 2025-04-12 06:00:05.538 INFO grpc_proxy.py:359 [alice] -- [Anonymou
s_job] ReceiverProxy binding port 8000, options: (('grpc.enable_retries', 1), ('grpc.so_reuseport'
, 0), ('grpc.max_send_message_length', 524288000), ('grpc.max_receive_message_length', 524288000),
 ('grpc.service_config', '{"methodConfig": [{"name": [{"service": "GrpcService"}], "retryPolicy":
{"maxAttempts": 5, "initialBackoff": "5s", "maxBackoff": "30s", "backoffMultiplier": 2, "retryable
StatusCodes": ["UNAVAILABLE"]}}]}'))...
(ReceiverProxyActor pid=66123) 2025-04-12 06:00:05.542 INFO grpc_proxy.py:379 [alice] -- [Anonymou
s_job] Successfully start Grpc service without credentials.
2025-04-12 06:00:06.626 INFO barriers.py:333 [alice] -- [Anonymous_job] SenderProxyActor has succe
ssfully created.
2025-04-12 06:00:06.627 INFO barriers.py:520 [alice] -- [Anonymous_job] Try ping ['bob'] at 0 atte
mp, up to 3600 attemps.
2025-04-12 06:00:09.632 INFO barriers.py:520 [alice] -- [Anonymous_job] Try ping ['bob'] at 1 atte
mp, up to 3600 attemps.
(pyu_fn pid=20400) 2025-04-12 06:00:11,976,976 INFO [xla_bridge.py:backends:863] Unable to initial
ize backend 'cuda':
(pyu_fn pid=20400) 2025-04-12 06:00:11,976,976 INFO [xla_bridge.py:backends:863] Unable to initial
ize backend 'rocm': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
(pyu_fn pid=20400) 2025-04-12 06:00:11,978,978 INFO [xla_bridge.py:backends:863] Unable to initial
ize backend 'tpu': INTERNAL: Failed to open libtpu.so: libtpu.so: cannot open shared object file:
No such file or directory
(pyu_fn pid=20400) [2025-04-12 06:00:12.027] [warning] [openssl_factory.cc:83] Yacl has been confi
gured to use Yacl's entropy source, but unable to find one. Fallback to use openssl's default entr
opy srouce
(pyu_fn pid=20400) [2025-04-12 06:00:12.029] [warning] [openssl_factory.cc:83] Yacl has been confi
gured to use Yacl's entropy source, but unable to find one. Fallback to use openssl's default entr
opy srouce
计算结果为： [{'deposit': array(990.25, dtype=float32), 'id': array(12345, dtype=int32)}, {'deposi
t': array(99990.25, dtype=float32), 'id': array(12345, dtype=int32)}]
(SPURuntime pid=66288) 2025-04-12 06:00:16.639 [warning] [openssl_factory.cc:OpensslDrbg:83] Yacl
has been configured to use Yacl's entropy source, but unable to find one. Fallback to use openssl'
s default entropy srouce [repeated 2x across cluster] (Ray deduplicates logs by default. Set RAY_D
EDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/r
ay-logging.html#log-deduplication for more options.)

以上代码用在测试中，在生产环境是不是也可以这么用，这么用的话在alice端会不会可以直接获取到bob的原始数据还不知道，请过路的大佬有时间也帮忙解答一下，我还在学习中~~~~

[wangzul]

可以这样用：https://www.secretflow.org.cn/zh-CN/docs/secretflow/v1.14.0b0/developer/benchmark/psi_benchmark#ray-header