Skip to content

Can't talk to partitions that change their remoteHandle #33

Open
@marmbrus

Description

@marmbrus

either due to change of ip address or actorNumber. Note, this reproduction kinda of sucks. I'll come up with something cleaner after sigmod.

Loading setup.scala...
import deploylib._
import deploylib.mesos._
import deploylib.ec2._
import edu.berkeley.cs.scads.comm._
import edu.berkeley.cs.scads.storage._
import edu.berkeley.cs.scads.piql._
import edu.berkeley.cs.avro.marker._
import edu.berkeley.cs.avro.runtime._
import java.io.File
import edu.berkeley.cs.scads.perf._
import scadr.cardinality.CardinalityExperiment
scheduler: deploylib.mesos.LocalExperimentScheduler = deploylib.mesos.LocalExperimentScheduler@94265ae
classpath: Seq[edu.berkeley.cs.scads.comm.S3CachedJar]
INF [20101025-13:05:17.791] comm: Opening Zookeeper Connection to mesos-ec2.knowsql.org with timeout 30000
INF [20101025-13:05:17.906] mesos: Registered SCADS Framework. Fid: 201010252005-0-0015
ec2zoo: edu.berkeley.cs.scads.comm.ZooKeeperProxy#ZooKeeperNode = <znode path:/, data: '', children: List(scads, zookeeper)>

Welcome to Scala version 2.8.0.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_20).
Type in expressions to have them evaluated.
Type :help for more information.

scala> ScalaEngine.main("zk://mesos-ec2/scads/laptopcluster", new File("localcluster"), None, false)
INF [20101025-13:05:22.154] comm: Opening Zookeeper Connection to mesos-ec2 with timeout 30000
INF [20101025-13:05:22.458] storage: Opening BDB Environment: localcluster, allowCreate=true
INF [20101025-13:05:22.458] storage: cacheSize=0
INF [20101025-13:05:22.458] storage: txnNoSync=false
INF [20101025-13:05:22.458] storage: txnWriteNoSync=false
INF [20101025-13:05:22.458] storage: {je.env.isTransactional=true, je.maxMemoryPercent=80}

WARNING: Could not find SCADS configuration file to load.
Options are:
(1) Set environmental variable SCADS_CONFIG to the config file path
(2) Set system property scads.config to the config file path
Using default values for config.

INF [20101025-13:05:24.354] comm: Using handler impl class: edu.berkeley.cs.scads.comm.DefaultNioChannelManager
INF [20101025-13:05:24.498] comm: Now serving on: 0.0.0.0/0.0.0.0:9000
INF [20101025-13:05:24.499] comm: Listener started on port: 9000
INF [20101025-13:05:24.771] storage: Recreating partition idxUsersTarget0000000000 from request {"namespace": "idxUsersTarget", "startKey": null, "endKey": null}
INF [20101025-13:05:26.618] storage: Recreating partition subscriptions0000000000 from request {"namespace": "subscriptions", "startKey": null, "endKey": null}
INF [20101025-13:05:28.393] storage: Recreating partition tags0000000000 from request {"namespace": "tags", "startKey": null, "endKey": null}
INF [20101025-13:05:30.174] storage: Recreating partition thoughts0000000000 from request {"namespace": "thoughts", "startKey": null, "endKey": null}
INF [20101025-13:05:31.989] storage: Recreating partition users0000000000 from request {"namespace": "users", "startKey": null, "endKey": null}
res0: edu.berkeley.cs.scads.storage.StorageHandler = <CounterID: 0, EnvDir: /Users/marmbrus/Workspace/radlab/scads/perf/localcluster, Handle: {"num": 0}@dhcp-44-134.eecs.berkeley.edu:9000>

scala> scadr.cardinality.ThoughtStreamClient()
:38: error: not enough arguments for method apply: (numClients: Int,executorClass: String,iterations: Int,threads: Int,runLengthMin: Int)edu.berkeley.cs.scads.perf.scadr.cardinality.ThoughtStreamClient in object ThoughtStreamClient.
Unspecified value parameters numClients, executorClass.
scadr.cardinality.ThoughtStreamClient()
^

scala> scadr.cardinality.ThoughtStreamClient(1, "edu.berkeley.cs.scads.piql.BulkParallelExecutor", 50, 1)
res2: edu.berkeley.cs.scads.perf.scadr.cardinality.ThoughtStreamClient = {"numClients": 1, "executorClass": "edu.berkeley.cs.scads.piql.BulkParallelExecutor", "iterations": 50, "threads": 1, "runLengthMin": 5}

scala> res2.run(res0.root)
INF [20101025-13:06:54.315] perf: Waiting for cluster to be ready
INF [20101025-13:06:54.691] perf: Begining iteration 1
INF [20101025-13:06:54.709] comm: Opening Zookeeper Connection to mesos-ec2.knowsql.org:2181 with timeout 30000
INF [20101025-13:07:01.303] comm: Connecting to dhcp-44-134.eecs.berkeley.edu/128.32.44.134:9000
INF [20101025-13:07:01.304] comm: Not connected, deferring connection
CRI [20101025-13:07:01.392] comm: Got message for an unknown service: {"num": 8}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions